cupsd.conf

We now drop the assumption that we can provide the scheduler with perfect prediction of the tasks’ peak requirements which makes the analysis more difficult but much more realistic. We discuss the impact of the various predictors on the scheduling decisions. Figure 5.5 presents the total number of active machines during the 24 hours of our simulation. The number of active machines is updated every 5 minutes. A machine is

considered active if itsCPUorRAMutilisation is higher than 0%. The figure shows the

results for two of the proposed prediction techniques. Results from the ad-hoc predic- tion technique, namely U L%, are not presented due to limitations of our tool when han-

dling huge numbers of evictions. We ran U L%for 9 hours and noticed almost the same

number of evictions than when running Random forest for 24 hours. Large number of evictions directly impact the task waiting time, number of active machines, and the

levels of utilisation of the machines, therefore the U L%predictor would undoubtably

have produced the worst results. Moreover, the figure also presents results for a perfect predictor, namely Actual Peak, and the user limit. The Actual Peak represents a theo- retical scenario, where the predictor knows the actual maximum resource consumption for all the incoming tasks. On the other hand, one can think about the user limit as a practical baseline since it is the only information accessible at scheduling time in the original description of the problem.

0 5 10 15 20 25 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 Time (hour) Active machines Random forest Multiple LR User limit Actual peak

Figure 5.5: Active machines (considering windows of 5 minutes)

1 2 3 4 5 6 0 10 20 30 40 50 60 70 80 90 Window step

CPU consumption (MU)

Random forest Multiple LR User limit Actual peak

1 2 3 4 5 6 0 10 20 30 40 50 60 70 80 90 Window step

RAM consumption (MU)

Figure 5.6: Cumulative averageCPU(left) andRAM (right) utilisation

given time. We observe that Random forest is the second predictor in terms of lowest number of active machines. In comparison to U L, random forest uses on average 671.9 fewer machines, with a maximum difference of 3,337 fewer machines. Moreover, in 18.9 hours out of the 24 hours studied, Random forest used fewer machines than U L. The figure also shows that Random forest and Multiple LR have a more consistent number of active machines than U L. For instance, we noticed two times periods in which both predictors significantly outperform U L, namely the range between 12.83 and 16.75 hours, and between 18.41 and 19.08 hours, in those two periods the average difference in terms of active machines between Random forest and U L is more than 2,300 machines.

Nevertheless, in contrast to their results in MAE and RMSE, we notice that the other

two predictors, U L% and Multiple LR, are in most cases outperformed by U L. We

observe that U L uses on average 1,511 fewer machines than Multiple LR. These results suggests that MAE and RMSE are not enough to assess the impact of the predictors when used for scheduling purposes. The high number of machines utilized by Multiple LR is caused by generally under estimating the actual peak of a task, which causes the number of evictions and thus the number of machines to increase. However, the time for which these large machines are active is in many cases only a few minutes, since the majority of the tasks that are evicted are tasks with short duration.

We present in Figure 5.6 the cumulative average utilisation of every 4 hours for all the

machines in the cluster. We measure the CPUandRAMconsumption and observe that

in general Random forest achieves higher average utilisation levels than U L. Random forest performs best in increasing the average utilisation in 5 out of 6 of the cumulative

time windows, bringing up to 10.08% more utilisation for CPUand 10.39% for RAM

when compared to U L. Furthermore, we notice that the actual peak predictor has, in the last two measurements, lower levels of utilisation than random forest or the default

5. ONLINECONSOLIDATION WITH

UNCERTAINTASKSIZES 5.5 Experiments

Table 5.2: Waiting time and number of evictions for tasks of different priorities. Priorities

free normal high

Multiple LR WT≤2s 96.43% 98.44% 95.23% Evictions 36467 78065 1633 mean WT 0.0013s 0.0001s 0.0010s Random Forest WT≤2s 97.95% 98.98% 97.20% Evictions 17834 34209 169 mean WT 0.0013s 0.0004s 0.0008s User Limit WT≤2s 98.74% 99.46% 99.60% Evictions 11645 24606 25 mean WT 0.0016s 0.0007s 0.0020s

user limit predictor. The reason is that the two predictors and the U L have to deal with evictions, which produces an increase on the overall consumption of the cluster.

The figure also shows that in the case ofCPU, there is a maximum difference of 20.17%

for Multiple LR with regards to U L. In the case ofRAM, the results are similar, namely a maximum difference of 21.36% for Multiple LR, when compared to U L. Moreover, we observed large standard deviations for the all the studied predictors. For instance,

in the case of Random forest the average standard deviation forCPUis 21.68% and for

RAM22.29%.

In Table 5.2 we study the task waiting time and the number of evictions, while consid- ering the priorities of the tasks. We highlighted with bold font the best performance for each of the cases considered. When studying the percentage of tasks with a wait- ing time (W T ) lower or equal to 2 seconds, we noticed a difference between Random forest and U L of less than 0.8%, for tasks of low and normal priority. In the case of tasks with high priority, the difference is 2.26%. Moreover, when using U L, the sched- uler produces the lowest number of evictions regardless of the priority of the task, this is expected since users tend to over-request the amount of resources that they need. Random forest and Multiple LR produce the lowest average waiting time for tasks that were not evicted, however the difference with regards U L is very small, since the maximum average W T is only 0.0013 seconds. Furthermore, given the high number of active machines and their lower levels of utilisation, using Multiple LR yields the highest number of evictions and lowest percentage of tasks being scheduled in less than 2 seconds.

The waiting time for evicted tasks is high for all the predictors. For instance, the average W T for evicted tasks with Normal priority is almost 10 minutes when the scheduler uses the Random forest predictor. The reason this large W T is that when

�

Figure 5.7: Aggregated CPU peaks.

our scheduler evicts a task, it only considers the task’s priority, regardless of how long a task has been running.

To better understand the performance of the predictors, we aggregate the peak CPU

utilisation for all the tasks with the assumption that the waiting time is 0 seconds. Figure 5.7 presents the results for all the prediction techniques in terms of aggregated

CPUpeaks. One can interpret the cumulated actual peaks as the amount of work that

the cloud system has to tackle. For each predictor considered here, we report the cumulated peaks as expected by the predictor. Visually, the closer a predictor is to the actual workload, the better the predictor is. We notice that random forest is the predictor that resembles the most to the Actual Peak. Moreover, we notice that Multiple

LR and U L% reach very low levels of utilisation when compared to the Actual Peak.

The random forest load predictor is the best candidate for accurate forecasts of peak resource utilisation.

5. ONLINECONSOLIDATION WITH

UNCERTAINTASKSIZES 5.5 Experiments