2.7.1
The Classad Language
Classad (an abridgment of Classified Advertisements) is a functional language which was developed in the Condor Project (2.2.3). It is now an important language in distributed computing, especially where resource allocation and matching is needed. An ‘advertisement’ (also known as a Classad) is the working unit of the language. Classads are analogous to the ‘for sale’ and ‘wanted’ classified adverts seen in newspa-
pers. Providers specify what they have to offer and requesters specify what they want to procure. For work allocation, an advertisement consists of one or more uniquely namedattribute andexpression pairs (or simply expressions), each of which describes properties and requirements of resources and jobs. Both parties list requirements (or constraints) that need to be satisfied, allowing both providers and requesters to ex- press constraints on each other. For illustration, figure 2.3 shows an example of two simple ‘used car’ classified advertisements and their equivalent Classads which express both what is offered and required by both parties.
For Sale: Used car. Petrol engine, 45,000 miles, Price, €10,000 Cash payment only.
Wanted: Used car, Petrol or diesel, Mileage under 50,000 only Willing to pay cash, €10,000
[Type = Seller; Fuel = Petrol; Mileage = 45000; Cost = 10000; Payment = Cash;
Requirements = other.Type == Buyer && other.Payment = Cash;
Rank = other.PriceOffer;]
[Type = Buyer; PriceOffer = 10000; Payment = Cash;
Requirements = other.Type == Seller && other.PriceCost <= 10000 && other.Fuel == Petrol || Diesel && Mileage <= 50000;
Rank = other.Cost;]
Classad Expression for Seller Classad Expression for Buyer
Seller
Potential buyer
Figure 2.3: Simple used car advertisements with equivalent Classad expressions.
The Classad language matches advertisements in a pairwise fashion. The output of this matching process can be either one of TRUE, FALSE, UNDEFINED or ER- ROR. The undefined result is the outcome of an attempt to match two ads where an attribute is defined in one message but missing in the other. An ERROR outcome may be (among other things) the result of an incorrectly defined attribute. Where there are a number of potential matches, these can be ranked based on values in one or more attributes. The attribute to be ranked is specified by the requester; for
example, in figure 2.3, the buyer may look to rank matches in order of ascending mileage.
Classad does not have a fixed schema and attributes can be given any name. The number of attribute and expression pairs is extensible. For matching, the required attributes are included in expressions asRequirements along with an attribute which is to be ranked. Classads from both providers and requesters should include these two expressions [110]. For other examples of Classad expressions, see appendix N, page 219.
Generality of Classad: Classad can be used as ageneral matching mechanism and is not bound to any type of resources or services being described. Attribute-expression pairs can be used to describe anything which can be described using literals such as strings, integers, etc. This freedom has allowed Classad to be used in other areas, for example in the development of Social Grid Agents (see chapter 3), and within a large number of grid projects [55].
Classad, when taken in toto, is a more versatile and richer language than its use in matching key pairs suggests. Classad expressions can also be used to contain ‘payload’ text as content. It can also be structured to take on the semantic-content form as seen in some agent communication languages [67, 63]. In the context of using Classad as an agent language, these are discussed further in sections 3.1.4 and 3.3.
2.7.2
Resource Allocation in Grids
The current allocation approach used for grid resources uses mainly Boolean matching and ranking is provided for matches where there are more than one criterion.
In the context of sharing and aggregating grid resources, CPU and storage quotas are agreed between grid site managers. These are then encoded in rules within the Workload Management System (WMS) in Grid infrastructure (see figure 2.2). Personnel representing virtual organisations may request changes in quotas as needs arise. National level operations centers can make final decisions on these allocations. These decisions will be influenced by contractual obligations between institutions,
suitability of work and the original reasons behind the provisioning of the resources. As a further example, SweGrid (the Swedish National Grid Initiative) and its subsidiary SNAC (Swedish National Allocations Committee) is a committee which meets to evaluate and make decisions on large allocation of the resources which it controls. In summary, some criteria used are... “The evaluation is based on scientific merit, need for the resources, feasibility of efficient use of the requested resources, and impact” [27]. The outcome of this is added an automatic fair-share allocation system within their infrastructure.
Job Description Language
At the level of allocating individual grid jobs to actual grid nodes, the European Grid Infrastructure (EGI) (section 2.7.1) employs a job description language (JDL) and a resource broker based on the Classad matching tool. This approach derives from the EU DataGrid project [28], further refined in the EGEE, EGEE-II and EGEE-III projects [29]. The language consists of a set of pre-defined attributes which describe both jobs and resources. Examples of these attributes areexecutable name,JobType,
ExpiryTime and Input- and OutputSandbox to specify the files needed to start the jobs and obtain their output respectively. The GLUE schema, which is an extension of the Classad language [55] is a specification for defining attributes used to describe grid resources and services.
2.7.3
Resource Allocation in Volunteer Computing
At the level of the individual owner of a PC, allocation ‘policy’ is ultimately dictated by that individual. No formal policy may exist, but it is likely that the owner of the resource will have an affinity towards the work of the chosen project.
In the World Community Grid initiative (2.4.5), administrators take a close inter- est in the type of project to which it allocates resources. They also vet project aims and provide technical, security and reliability guidance for application code used. In keeping with the social and economic consideration of volunteers outlined in sec- tion 2.4.1, only projects which are ‘not for profit’ and which have a ‘humanitarian
orientation‘ are allocated volunteered resources by WCG administrators [20]. This allocation process is carried out by WCG personnel who meet on a periodic basis1.
2.7.4
Resource Allocation in Commercial Cloud
Full free-market forces such as supply and demand, cost effectiveness and ‘return on investment’ apply in allocation of commercial cloud resources. As the cloud mar- ket and the technology used matures, the barrier to entry for providers is lowering. Similarly, usage of cloud resources is likely to get easier as user interfaces and APIs evolve. Open-source cloud software such as OpenNebula [30] and Eucalyptus [31] allow individuals and small companies to investigate using privately owned clouds by way of introduction to the technology. This means that in the future, a quite fluid cloud resource market may evolve, and that computing resources will become even more a commodity.