Provisioning implies the deployment and configuration of virtual appliances in a cloud environment. CMPs are those above-presented permit serf-service provisioning, although there is an increasing number of tools automatizing the process (see 1.4.3.4 introduction’s section). For the infrastructures presented here, provisioning is managed by one of the two tools hereafter described.
〕 PMES
Programming Model Enactment Service (PMES) allows the remote deployment and job management of computing virtualized services in distributed infrastructures. PMES offers a standard REST interface to manage VM provisioning in a uniform and elastic manner, hiding in this way the heterogeneities of the underlining cloud stack. Indeed, it goes beyond and
controls the full-service lifecycle of pre-emptible stateless VMs that act as compute units. Sequentially, stages are the (i) deployment of the appliance, (ii) stage in data, (iii) execution invocation and monitoring of the application of interest, (iv) stage out data, and eventually (v) un-deployment of the VM(s). Cloud provisioning tasks are undergone via the cloud standard OCCI (below detailed at 3.2 Open standards), which mediates PMES interactions with the cloud infrastructure. Thus, OCCI compliant CMPs are a requirement. The communication of PMES with the transient VMs is enabled via SSH using a pair of public keys adequately contextualized at boot time. FTP is used for data transferences, which permits not only the dispatching of input and output files, but also the transference of a JAR file containing the actual application to be executed. As such, VMs can automatically be provisioned with the software at boot time.
Here, PMES is installed in two different cloud middlewares, OpenNebula and OpenStack. For each IaaS a stand-alone server is installed as a permanent virtual machine, accessible for outside the local VLAN. Two PMES components are part of that server:
- PMES service: deployed as a Tomcat 7 [203] application
- PMES dashboard service: NodeJS [204] application deployed using mp2 [205] The installation requires the specification of the compute node resources available and importantly, the cloud rOCCI endpoint. Full documentation of the installation process can be found annexed at 8.6.1 PMES documentation. PMES service exposes two APIs implementations, one SOAP-based (documented online [206]), used in the transPLANT infrastructure, the other enabled in REST and used in openVRE platforms (documentation annexed at 8.6.2PMES documentation). APIs are compliant with two well-known open standards inherited from grid computing, BES and JSDL (further discussed below at 3.2). 〕 OneFlow
OneFlow is the key component of OpenNebula to build scalable services. It creates, controls and monitors services – composed by a single VM or a group of them – self-provisioning them in basis to (i) deployment dependencies between the group of VMs, (ii) elasticity policies. These policies establish the triggering parameters that make OneFlow scale up or down the number of running VMs for a given service. Interestingly, the CPU or MEM load of the VMs can be set as triggering parameters, thus, in front of load burst, OneFlow automatically makes available extra instances. OneFlow is installed as part of OneNebula, implemented as one of its add-ons.
Contextualization
A context is a small, usually human-readable, snippet that is used to apply a role to a VM. It contains the relevant information from the cloud manager to configure that role at the “first
boot”. Hence, a context allows having a single VMI backing many different VM instances as long as the instances vary on settings able to be configured in the context. Considering we have a catalog of VMIs pre-packing bioinformatics tools, we can concurrently instantiate a VMI several times, each contextualized with the particular needs of that run. Typically, contextualization is useful for setting configurations not known until instantiation, like DNS, hostname, private information like user’s SSH keys or personal data volumes, or software that changes frequently with discrete user-defined commands.
CONTEXT=[ PUBLIC_IP="192.168.11.247”, . DNS="192.168.0.202", GATEWAY = "192.168.11.202", MAC = "02:00:c0:a8:0b:f7", MASK = "255.255.255.0", SEARCH_DOMAIN = "vm.mmb.pcb.ub.es",
SSH_KEY="ssh-rsa AAAAB3NzaC1yc2EAAAA[...]Q/I0w== pmes@mmb", HOSTNAME = "mg-192.168.11.247", USER_DATA=" #cloud-config packages: [] mounts: - [vdc,none,swap,sw,0,0] runcmd:
- echo 'Success' | wall”
]
Snippet 3.2 : Example of OpenNebula context information prepared to be consumed by cloud-init
CMPs, either OpenNebula or OpenStack, are responsible to pass the context information into the newly created VM. Each CMP enables different methods to do so. OpenNebula parametrize VMs by a iso 9660 CD-ROM image, which contains a shell script (context.sh) with custom variables defined on virtual machine start. Snippet 3.2 is an example of that information transmitted by OpenNebula and ready to be consumed by the contextualization package installed at the VM. There are no fixed contextualization variables, and are in charge of integrating them and locally configuring the instance accordingly. OpenStack sends the information to a metadata server configured to serve up vendor data in a JSON object. Contextualization packages will reach it from there.
During the present work, two different contextualization systems are used to achieve very similar parameterization tasks, yet One-context is specific for instances deployed with OpenNebula, while Cloud-init supports several CMPs.
〕 One-context
One-context is an add-on that provides contextualization packages for most Linux distributions (and, other Unix-like) on guest VMs or Linux containers running in the OpenNebula cloud. Packages are able to read context variables and prepare, accordingly, the networking in the running guest operating system, configure SSH keys, set passwords, run custom start scripts, and many others functionalities. Some of these functionalities are
OpenNebula-specific, like passing token-based credentials of OneGate, an OpenNebula module to enable authentication among VMs.
We used One-context as part of the transPLANT VMIs (see 4.2. Results). We installed the corresponding DEB package (v. 5.7.0) into Ubuntu 14.04 LTS images, before importing them into the OpenNebula as QCOW2 images. On the other hand, OpenNebula templates need to be configured as Snippet 3.2 exemplifies. The system is easy to be installed and configured, and USER_DATA is passed as opaque key-value metadata to the VM with no major restrictions. However, One-context can only be used for OpenNebula IaaS.
〕 Cloud-init
Although few cloud open standards are still well-established, cloud-init is becoming the standard de facto for contextualization. Similarly to One-context, Cloud-init is used for actions on “first boot”, pre- and post- networking, and USER_DATA section permits also some post-boot configuration by means of YAML scripts. However, the most remarkable feature of Cloud-init is its interoperability. It provides connectors (“datasources”) for the most popular CMPs, including OpenNebula, OpenStack, Amazon EC2 or Azure.
As such, VMIs with Cloud-init are ready to be portable among different cloud providers with no need of further configurations or conversions, a relevant feature exploited at MuG infrastructures, as discussed at 4.3. Results. Cloud-init (v. 0.7.5.) is installed in MuG VMIs and configured accordingly, choosing the correct “Datasource” and configuring some network details as Cloud-init demands them, like determined interface names or settings.