Servers can be categorized as either iterative, concurrent, or reactive. The primary trade-offs in this dimension involve simplicity of programming ver-sus the ability to scale to increased service offerings and host loads.
Iterative servers handle each client request in its entirety before servic-ing subsequent requests. While processservic-ing a request, an iterative server therefore either queues or ignores additional requests. Iterative servers are best suited for either
103
104 CHAPTER 5 Concurrency Design Dimensions
ITERATIVE/REACTIVE SERVER (2) CONCURRENT SERVER
Figure Iterative/Reactive versus Concurrent Servers
• Short-duration services, such as the standard Internet ECHO and DAYTIME services, that have minimal execution time variation or
• Infrequently run services, such as a remote file system backup ser-vice that runs nightly when platforms are lightly loaded
Iterative servers are relatively straightforward to develop. Figure (1) illustrates how they often execute their service requests internally within a single process address space, as shown by the following pseudo-code:
void iterative_server ( )
{ initialize listener endpoint(s)
for (each new client request) {
retrieve next request from an input source perform requested service
if (response required) send response to client }
Due to this iterative structure, the processing of each request is serial-ized at a relatively coarse-grained level, for example, at the interface be-tween the application and an OS synchronous event demultiplexer, such as
() or However, this coarse-grained
level of concurrency can certain processing resources (such as multiple CPUs) and OS features (such as support for parallel DMA transfer to/from I/O devices) that are available on a host platform.
Section Iterative, and Reactive Servers 1O5
Iterative servers can also prevent clients from making progress while they are blocked waiting for a server to process their requests. Excessive server-side delays complicate application and middleware-level retrans-mission time-out calculations, which can trigger excessive network traffic.
Depending on the types of protocols used to exchange requests between client and server, duplicate requests may also be received by a server.
Concurrent servers handle multiple requests from clients simultane-ously, as shown in Figure (2). Depending on the OS and hardware platform, a concurrent server either executes its services using multiple threads or multiple processes. If the server is a single-service server, mul-tiple copies of the same service can run simultaneously. If the server is a multiservice server, multiple copies of different services may also run simultaneously.
Concurrent servers are well-suited for services and/or long-duration services that require variable amounts of time to execute. Unlike iterative servers, concurrent servers allow finer grained synchronization techniques that serialize requests at an application-defined level. This de-sign requires synchronization mechanisms, such as semaphores or mutex to ensure robust cooperation and data sharing between processes and threads that run simultaneously. We examine these mech-anisms in Chapter 6 and show examples of their use in Chapter 10.
As we'll see in Section 5.2, concurrent servers can be structured various ways, for example, with multiple processes or threads. A common con-current server design is where a master thread spawns a separate worker thread to perform each client request concurrently:
void {
initialize listener endpoint(s) for (each new client request) {
receive the request
spawn new worker thread and pass request to this thread }
The master thread continues to listen for new requests, while the worker thread processes the client request, as follows:
void worker thread () {
perform requested service
106 Concurrency Design Dimensions
if (response required) send response to client terminate thread
It's straightforward to modify this thread-per-request model to support other concurrent server models, such as
void master thread ( ) {
initialize listener endpoint(s) for (each new client connection) {
accept connection
spawn new worker thread and pass connection to this thread
In this design, the master thread continues to listen for new connections, while the worker thread processes client requests from the connection, as follows:
void worker_thread ( )
for (each request on the connection) { receive the request
perform requested service
if (response required) send response to client }
Thread-per-connection provides good support for of client requests. For instance, connections from high-priority clients can be as-sociated with high-priority threads. Requests from higher-priority clients will therefore be served ahead of requests from clients since the OS can preempt threads.
Section 5.3 illustrates several other concurrent server models, such as thread pool and process pool.
Reactive servers process multiple requests virtually simultaneously, al-though all processing is actually done in a single thread. Before mul-tithreading was widely available on OS platforms, concurrent processing was often implemented via a synchronous event demultiplexing strategy where multiple service requests were handled in round-robin order by a single-threaded process. For instance, the standard X Windows server op-erates this way.
Section Iterative, Concurrent, and Reactive Servers 107
A reactive server can be implemented by explicitly time-slicing atten-tion to each request via synchronous event demultiplexing mechanisms,
such as select () and () described in
Chap-ter 6. The following pseudo-code illustrates the typical style of program-ming used in a reactive server based on select
void reactive_server () {
initialize listener endpoint(s) // Event
{
select () on multiple endpoints for client requests for (each active client request) {
receive the request
perform requested service
if (response is necessary) send response to client
Although this server can service multiple clients over a period of time, it's fundamentally iterative from the server's perspective. Compared with tak-ing advantage of full-fledged OS support for multithreadtak-ing, therefore, ap-plications developed using this technique possess the following limitations:
• Increased programming complexity. Certain types of networked applications, such as servers, are hard to program with a reac-tive server model. For example, developers are responsible for yielding the event loop thread explicitly and saving and restoring context information manually. For clients to perceive that their requests are being handled concurrently rather than iteratively, therefore, each request must execute for a relatively short duration. Likewise, long-duration operations, such as downloading large files, must be programmed explicitly as finite state machines that keep track of an object's processing steps while reacting to events for other objects. This design can become unwieldy as the number of states increases.
• Decreased dependability and performance. An entire server pro-cess can hang if a single operation fails, for example, if a service goes into an infinite loop or hangs indefinitely in a deadlock. Moreover, even if the entire process doesn't fail, its performance will degrade if the OS blocks the whole process whenever one service calls a system function or incurs a page fault. Conversely, if only nonblocking methods are used, it can be
108 CHAPTER 5 Concurrency Design Dimensions
hard to improve performance via advanced techniques, such as DMA, that benefit from locality of reference in data and instruction caches. As dis-cussed in Chapter 6, OS multithreading mechanisms can overcome these performance limitations by automating preemptive and parallel execution of independent services running in separate threads. One way to work around these problems without going to a full-blown concurrent server so-lution is to use asynchronous I/O, which is described in Sidebar
Sidebar Asynchronous I/O and Proactive Servers
Yet another mechanism for handling multiple I/O streams in a single-threaded server is asynchronous I/O. This mechanism allows a server to initiate I/O requests via one or more I/O handles without blocking for completion. the OS notifies the caller when requests are com-plete, and the server can then continue processing on the completed I/O handles. Asynchronous I/O is available on the following OS plat-forms:
• It's supported on Win32 via overlapped I/O and I/O completion ports
• Some platforms implement the aio_* family of asynchronous I/O functions (POS95,
Since asynchronous isn't implemented as portably as multithread-ing or synchronous event demultiplexmultithread-ing, however, we don't consider it further in this book.
Asynchronous I/O is discussed in (SH) when we present the ACE Proactor framework, which implements the Proactor (SSRBOO).
This pattern allows event-driven applications to demultiplex and dis-patch service requests efficiently when they are triggered by the completion of asynchronous operations, thereby achieving the perfor-mance benefits of concurrency without incurring certain of its liabilities, The ACE Proactor framework runs on Win32 and on POSIX-compliant platforms that support the aio_* family of asynchronous I/O func-tions.
Logging service For simplicity, the initial implementation of our net-worked logging service in Chapter 4 used an iterative server design.
Subse-Section 5.2 Processes versus Threads 1O9
MULTIPROCESSING
Figure 5.2: Multiprocessing versus Multithreading
quent chapters extend the capabilities and scalability of our logging server as follows: Chapter 7 extends the server to show a reactive style, Chap-ter 8 illustrates a concurrent style using multiple processes, and ChapChap-ter 9 shows several concurrent designs using multiple threads.