CONCLUSIONES Y RECOMENDACIONES

To examine the impact of SIP message processing and SIP operations including database lookups, we compare with the previous measurement using the echo server. Figure 4.1

compares the response times for the SIP REGISTER-200 OK transaction including the setup times and CPU utilization at 2,500 requests/second between the SIP server and echo server measurements described in Figure3.3. The SIP REGISTER-200 OK transaction has the same

0 0.5 1 1.5 2 transaction -based TCP persistent TCP w/open persistent TCP UDP 0 10 20 30 40 50 60 70 80 90 100

transaction response time (ms)

CPU usage (%)

response time for echo server CPU usage for echo server response time for SIP server CPU usage for SIP server

Figure 4.1: Average transaction response times (left axis) and peak CPU utilization (right axis) at 2,500 requests/second for echo server test and REGISTER-200 OK test

number of messages between the server and the clients as the echo server measurement. Both REGISTER request and 200 OK response were approximately 400 byte long while the echo server used messages approximately 500 byte long.

The increases indicate the cost of SIP operations and the difference in the software model of the two servers since the number of messages and transactions are the same. Interestingly, despite the fact that the SIP operations are the same for the four configuration, the increases of the response times differ by 0.4 - 1.2 ms for the four configurations while the increases of the CPU times are comparable at 15 - 18 percent. For example, the cost of establishing a new TCP connection, which is indicated in the difference in the transaction response time between the two persistent TCP configurations, is 0.4 ms on the SIP server, while it is 0.2 ms on the echo server. Thus, we deduce that these increased response times are caused by the difference in the software model of the two servers: the echo server uses a single process and a single thread model, while the SIP server runs a single process and multiple threads in a thread pool model. This difference in the software model is the natural consequence of the difference in server applications between the SIP server and the echo server. SIP

Figure 4.2: Transaction response times as a function of the sending rate for REGISTER-200 OK test for TCP and UDP

servers need to perform SIP operations, which are heavier than creating a message copy on the echo server, and achieve high performance with concurrent threads or processes. The sipd software model uses multi-threads in the thread pool model before improving server performance using the process pool model [Singh and Schulzrinne, 2005]. Sipd assigns available threads from a thread pool to waiting tasks in a queue. All the tasks for handling SIP request or response messages are processed in this manner except a task for reading a socket buffer and parsing a message over TCP. This task is processed by a thread that is generated upon request, not assigned from the thread pool, since the task lasts longer and may exhaust threads in the thread pool.

Similar to the response times, sustainable request rates also differ across the four con- figurations, namely, the three TCP configurations and the UDP setting. Figure 4.2 com- pares the transaction response times at various request sending rates at 100 percent success rate. The sustainable request rates are 2,900, 3,300, 4,100, and 5,300 requests/second each for the transaction-based TCP configuration, for the persistent TCP connections with the handshake processing, for the persistent TCP connections, and for UDP. These sustainable

Figure 4.3: Success rate, CPU usage and memory usage for persistent TCP

request rates are at least 27 times higher than the server design requirement described in Table2.2.

Whereas below 1,600 requests/second, the differences in the transaction response times remain constant, above that, the differences grow substantially. Interestingly, when the SIP server started to fail to handle SIP requests, the system resources had not been exhausted. As shown in Figures 4.3 and 4.4, for the persistent TCP configuration and UDP, CPU utilization was still below 40 percent and the usage of physical memory in resident set size (RSS) and virtual memory in virtual size (VSZ) was below 200 MB and below 800 MB, respectively. The results of the transaction-based TCP and a variant of the persistent TCP including opening TCP connections were similar although no charts are presented. Much earlier than exhausting CPU utilization or memory resources, sipd dropped requests based on the number of tasks waiting in the thread queue. When the number of the waiting tasks exceeds a pre-configured number, sipd invokes its overload control function to drop tasks. The sipd warning messages reported that the overload control function dropped 83 percent of requests for the persistent TCP configuration and 10 - 28 percent of requests for UDP. Thus, we have determined the bottleneck of sustainable request rate is the length limit of the thread queue, rather than memory usage or CPU utilization. More tasks and/or longer tasks required for handling TCP messages decrease the sustainable request rates.

Furthermore, the success rate for persistent TCP dropped precipitously at the load limit, whereas the rate for UDP gradually decreased. The difference between persistent TCP and UDP in dropping the success rate was caused by the difference in the timing of invoking the overload control function between TCP and UDP. Sipd determines if it can handle a UDP message immediately after parsing the first line of the message, enough to sort messages into requests and responses. It favors SIP responses over SIP requests under high loads. On the other hand, sipd first fully parses a TCP message by a thread generated upon request, not assigned from the thread pool, and then detects an overload in the thread queue. This difference is caused by the message orientation of these transport protocols. Since TCP does not preserve message boundaries, a receive buffer may contain only part of a SIP message or multiple SIP messages. Sipd needs to determine the end of a message by parsing the Content-Length header field. This suggests that if sipd can accelerate message

parsing over TCP, especially under high loads, the message parsing can be performed by a thread from the thread pool, resulting in a success rate can drop gradually, similar to UDP. To sort SIP messages, for example, to prioritize SIP responses or BYE requests, which have fewer subsequent messages under high loads, reading the first line of a received TCP stream can help accelerate determining the SIP message type. This message parsing without determining the end of a message does not properly work all the time, but is likely to parse the first line of a message especially at a SIP edge server, where messages are distributed over a large number of connections. Even if sipd fails in sorting messages, such as dropping a fragment of a message or multiple messages at once, it would be permissible under high loads.

It should be noted that this measurement experiment does not identify the cost of the SIP keep-alive messages, but implies that the SIP server needs to process the messages, which contain only four bytes, but arrive at a rate 3,000 requests/second, without overwhelming the thread queue or generating new threads for them. A possible solution is that a dedicated thread peeks at the size of a received message to dispatch tasks by distinguishing the keep- alive messages from regular SIP messages.

4.5

Measurement Results Using SCTP