In this section we present the integration of Cloud resources to the ASKALON re- source pool and evaluate the resulting enhanced system. We execute workflows on a hybrid environment build from Grid and Cloud resources.
We extended the ASKALON enactment engine to consider our Cloud extensions by transferring files and submitting jobs to Cloud resources using the SCP/SSH provider of the Java CoG kit [162]. We faced a technical problem with the latest Java CoG release that opens new threads for each SCP file transfer, which crashes the virtual machine after about 2500 file transfers and 5000 open threads. We therefore had to re-implement the file transfer functionality by ourselves to eliminate this bug.
We present experimental results using a real-world application in the Austrian Grid environment, extended with an own academic Cloud constructed using the Eucalyptus middleware [110] and Xen virtualization technology [29].
We selected for our experiments a scientific workflow application called Wien2k [17] as presented in Section 3.3. We have chosen a problem case (called atype) that we
8.0 and 9.0, which represents the number of planewaves that is equal to the size of
the eigenvalue problem (i.e. the size of the matrix to be diagonalised) referenced as problem complexity in this work.
We executed the workflow on a distributed testbed summarized in Table 4.4 and consisting of four heterogeneous Austrian Grid sites [161] and twelve virtual CPUs from an own “academic Cloud” called dps.cloud build using the Eucalyptus middle- ware [110] and the XEN virtualization mechanism [29]. We configured the dps.cloud resource classes to use one single core, while multi-core configurations were prohib- ited by a bug in the Eucalyptus software (planned to be fixed in the next release). We fixed the machine size of each Grid site to twelve cores each. This eliminates the vari- ability in the resource availability and makes the results across different experiments comparable.
We used a just-in-time scheduling mechanism that tries to map each activity onto the fastest available Grid resource. Once the Grid becomes full (because the size of the workflow parallel loops is larger than the total number of cores in the testbed), the scheduler starts requesting additional Cloud resources for execute in parallel the remaining workflow activities. Once these additional resources are available they will be used like Grid resources just with different job submission methods.
We define a new metricCT calledcost per unit of saved time ($/T)as ratio between
the total cost CostCloud of leasing the Cloud resources and the time saved by using
them:
CT =
CostCloud
TGrid−TGrid+Cloud
.
The termsTGrid andTGrid+Cloud can be calculated or approximated in three different
ways: real execution, simulation, or using prediction models. The goal of the dynamic provisioning is to maximize this metric by increasing the amount of saved time and
Grid site Location Cores used Cpu type GHz Memory / core
karwendel Innsbruck 12 Opteron 2.4 1024mb
altix1.uibk Innsbruck 12 Itanium 1.4 1024mb
altix1.jku Linz 12 Itanium 1.4 1024mb
hydra.gup Linz 12 Itanium 1.6 1024mb
dps.cloud Innsbruck 12 Opteron 2.2 1024mb
Figure 4.14: Wien2k execution time with 376 parallel activities. reducing cost of Cloud resources used.
Our goal was to compare the workflow execution for different problem sizes on the four Grid sites, with the execution using the same Grid environment supplemented by additional Cloud resources from dps.cloud. We executed each workflow instance for five times and reported the average values obtained. The runtime variability in the
Austrian Grid was less than5%, since the testbed was idle during our experiments and
each CPU was dedicated to running its activity with no external load or other queuing overheads.
Figure 4.14 and 4.15 shows the workflow execution times for367, respectively193
parallel activities in six different configurations showing a significant improvement in execution time when using Cloud resources.
Table 4.5 summarizes the workflow execution times for367, respectively193paral-
lel activities in six different configurations. The small, medium, and big configuration values represent a problem size parameter which influences the execution time of the parallel activities. The improvement when using Cloud resources compared to using
only the four Grid sites increases from a small1.08speedup for short workflows with
14minutes execution time, to a good 1.67speedup for large workflows with93min-
Figure 4.15: Wien2k execution time with 193 parallel activities.
Parallel Problem Grid Grid + Cloud Speedup Used instances Paid instances CT
activities complexity execution execution using Cloud Hours $ Hours $ $/min
193 Small (7.0) 874.66 803.66 1.09 2.7 0.54 12 2.4 2.02 193 Medium (8.0) 1915.41 1218.09 1.57 4.1 0.82 12 2.4 0.21 193 Big (9.0) 3670.18 2193.79 1.67 7.3 1.46 12 2.4 0.19 378 Small (7.0) 1458.92 1275.31 1.14 4.3 0.86 12 2.4 0.79 378 Medium (8.0) 2687.85 2020.17 1.33 6.7 1.34 12 2.4 0.22 378 Big (9.0) 5599.67 4228.90 1.32 14.1 2.81 24 4.8 0.11
Table 4.5: Wien2k execution time and cost analysis.
benefit much from the Cloud resources due to the high ratio between the smaller com- putation and the high provisioning and data transfer overheads. The main bottleneck when using Cloud resources is that the provisioned single core instances use separate file systems, which require separate file transfers to start the computation. In con- trast, Grid sites are usually parallel machines that share one file system across a larger number of cores which significantly decreases the data transfer overheads. Neverthe- less, for large problem sizes the Cloud resources can help to significantly shorten the workflow completion time in case Grids become overloaded.
Table 4.6 gives further details on the file transfer overheads and the distribution of activity instances between the pure Grid and the combined Grid-Cloud execution. The
Parallel File transfers Activities run
activities Total to Grid to Cloud Total on Cloud
378 2013 1544 469 (23%) 759 209 (28%)
193 1127 778 349 (31%) 389 107 (28%)
Table 4.6: Grid versus Cloud file transfer and activity instance distribution. file transfer overhead can be reduced by increasing the size of the used resource class (i.e. number of cores underneath one instance that share a filesystem and the input files for execution). But this may result in a lower resource allocation efficiency as the resource allocation granularity increases. We plan to investigate this trade–off in future work.
A characteristic of all Clouds that we surveyed in section 4.1 is that they charge the resource consumption based on hourly billing increments, and not based on one- second billing increments as assumed by the simulations performed in two related works [10, 36]. Table 4.5 shows that for our relatively short workflows below two hours, there can be a significant difference between the hourly and the one-second billing increment policies. This ratio is decreasing with the growing problem size
from4.4for the smallest workflow to1.64for the largest workflow.
The results show that the largest workflows are the most convenient to be scaled on additional Cloud resources and cost between $0.11–$0.19 per saved minute, while the small workflow exhibit high costs up to $2.02 per minute because of the hourly billing.
4.5 Related Work
Deelman et.al. [36] analyses the cost of Cloud storage for an image mosaic workflow and a possible on-demand calculation of the results. The work is based on an Amazon EC2 and S3 simulation rather than real execution. The computation cost model is based on one-second billing increment and the storage cost model on a byte-per-second billing increment, in contrast to real Cloud providers which charge based on hourly, respectively gigabyte-per-month billing increments.
Buyya et.al. [10] describes an approach of extending a local cluster with Cloud re- sources using two schedulers, one for the cluster and one for the Cloud, applying dif- ferent strategies. The possible benefit of not violating deadlines and achieving higher
cluster throughput is analyses. The system concentrates on clusters and does extend its scope to Grids or multiple Cloud providers. Their results are generated using sim- ulation and do not take the real speed of Cloud resources into account.
The work in [136] checks the usability of Cloud computing for scientific applica- tions using several special benchmarks and show that Cloud computing can be useful to scientific computing in general.
In [172], a framework to analyze the performance of Clouds is presented and the results encourage the usability of Clouds for loosely-coupled jobs like in workflows.