Throughput Response time DHR/BHR Resp. time/rate, client Resp. time/rate, server top1 ATS(slightly) Varnish(slightly) ATS(slightly) Varnish(slightly) almost the same
Iinc Varnish Varnish Varnish(slightly) Varnish Varnish
Table 4.7: Summary of results from the inc phase
4.5 Surrogate data results
In this section, the results of the surrogate data are discussed in the context of the preceding Polygraph results.
Best Effort Workload
Figure 4.35 shows the system CPU and memory usage during the Best Effort test. ATS uses both less CPU and a smaller percentage of memory throughout the test. Especially in terms of memory resources, ATS resource requirements remain relatively constant over the period of the test.
Figure 4.35: CPU and Memory Usage, Best Effort Workload
Varnish uses more CPU and more memory than ATS, and its CPU usage range is narrower than for ATS.
Figure 4.42 illustrates the Seekwatcher plots from the blktrace data recorded during the same experiment. Varnish sustains a higher I/O throughput rate from disk than ATS. The dip in the throughput graph occurs at the same time as the spike in the seek count plot. This is a logical result. These features occur during the experiment’s idle phase. Varnish may have entered a garbage collec-tion mode when the request rate dropped to 0, accounting for these artifacts.
Otherwise, the Varnish seek rate is quite low, indicating that it is either serving data from RAM or sending requests to the back end server.
4.5. SURROGATE DATA RESULTS
Figure 4.36: Disk I/O - mixed content type,increasing rate Single Content Workloads
Figure4.37illustrates the CPU and main memory use for the three single con-tent type workloads. ATS is relatively consiscon-tent in its memory usage; after a startup period, it uses between 25% and 34% of main memory across the various tests, with the maximum for HTML, the scenario for which it becomes satu-rated. Even after this occurs, ATS does not continually increase its memory use. In general, ATS is quite consistent in its memory usage across all of the experiments.
Varnish has completely flat memory usage for the HTML and Image test, al-though it uses more memory than ATS. For the Download content type, Varnish again uses more memory than ATS, and its usage oscillate very slightly during the course of the test.
For the Image and Download workloads, the CPU usage pattern for both servers is similar to that for the Best Effort workload, although the mean values shift as shown in Table 4.8. Note that the CPU percentage values are for 2 CPUs and so can exceed 100%. The idle phase for the Image experiment, during which server activity drops to zero, is excluded from the calculation of the mean.
Best Effort Image Download
ATS 20 7 102
Varnish 23 9 99
Table 4.8: Mean %CPU usage for various workloads (2 cpus)
The CPU usage for the HTML workload again indicates the stress experienced by the ATS server, evidenced by the numerous usage spikes (green line). In contrast, Varnish’s CPU usage remains steady at a quite low value of about 10% (again excluding the idle period).
4.5. SURROGATE DATA RESULTS
Figure 4.37: CPU/memory usage, single content type workloads
Figures 4.38 and 4.39 provide the Seekwatcher plots of the blktace data for the HTML and Image content types. In both cases, there is again an upsurge in seeks on the part of Varnish during the idle phase; in contrast, ATS seeks decrease almost to zero over the same period. In addition, Varnish exhibits another large spike later in both runs. This behavior is hard to explain, but it recurs during experiment repetitions and is something that happens periodically with Varnish. It is certainly worth additional investigation.
4.5. SURROGATE DATA RESULTS
Figure 4.38: Disk I/O - HTML content type
The two servers’ disk I/O throughput for these experiments show a very similar pattern to one another, although ATS again has a lower throughput level for both content types.
Figure 4.39: Disk I/O for Image content type
The Seekwatcher results for the Download content type are much more straight-forward. The seek rates and I/O throughput are both flat, and the two servers have essentially the same behavior profiles. This is in line with their similar results for throughput and close results for other metrics from the Polygraph tests.
Figure 4.40: Disk I/O - Download content type
4.5. SURROGATE DATA RESULTS
Mixed content results
Figure 4.41 shows the CPU and memory usage over the period of the mixed content workload experiment. The period prior to 70 minutes corresponds for the most part to the top1 phase. The two servers have essentially the same CPU usage over this period, with ATS being slightly more volatile. The memory usage again has ATS using several fewer percent of the CPU. These results are completely in agreement with those from the best effort and non-HTML single content tests.
Figure 4.41: CPU/memory usage, mixed content type workload
After minute 70, the inc phase begins, and the data changes significantly. Var-nish uses gradually increasingly more CPU and slightly more memory than during the top1 phase as the request rate rises.
ATS behavior is quite different. The CPU usage dramatically increases and os-cillates wildly as the request rate rises. At times, the system is close to running out of CPU resource. Memory use also increases, rising to a level significantly over that of Varnish.
Figure4.42 shows the Seekwatcher results for the same experiment. After the initial fill phase (from about the minute point), the two servers again have very similar results during the top1 phase. During the inc phase, Varnish throughput rises as it is capable of handling the increasing workload. In contrast, ATS’s throughput remains at a lower level corresponding to the fraction of the offered workload that it is capable of handling.
4.5. SURROGATE DATA RESULTS
Figure 4.42: Disk I/O, mixed content type workload
The Varnish seek data again includes one very large and one smaller spike during the inc phase. However, this characteristic if unexplained behavior does not affect its I/O throughput results or its ability to handle the workload. Clearly, this data feature must correspond to some routine function of Varnish.