Global resource attributes are cluster-wide resource attributes, such as available network bandwidth of a file server or the free disk space on a network-wide available file system.
Global resource attributes can also be associated with load reports if the corresponding load report contains the GLOBAL identifier, as described in "Configuring Load Parameters". Global load values can be reported from any host in the cluster. No global load values are reported by default, therefore there are no default global resource attributes.
Concrete values for global resource attributes are determined by the following items:
■ Global load reports.
■ Explicit definition in the complex_values parameter of the global host Note: Only the Shortcut, Relation, Requestable, Consumable, and
Default columns can be changed for the default resource attributes. No default attributes can be deleted.
■ In association with a particular host or queue and an explicit definition in the
corresponding complex_values lists.
Sometimes none of these cases apply. For example, a load value might not yet be reported. In such cases, the attribute does not exist.
2.1.4.10 Defining Consumable Resources
2.1.4.10.1 Introducing Consumable Resources Consumable resources, or consumables, provide an efficient way to manage limited resources. The complex also builds the framework for the system's consumable resources facility. The resource attributes that are defined in the complex can be attached to the global cluster, to a host, or to a queue instance. The attached attribute identifies a resource with the associated capability. Attribute definitions in the Grid Engine complex define how resource attributes should be interpreted. During the scheduling process, the availability of resources and the job requirements are taken into account. The Grid Engine system also performs the bookkeeping and the capacity planning that is required to prevent over-subscription of consumable resources.
Typical consumable resources include:
■ Available free memory ■ Unoccupied software licenses ■ Free disk space
■ Available bandwidth on a network connection
The definition of a resource attribute includes the following:
■ Name of the attribute
■ Shortcut to reference the attribute name
■ Value type of the attribute, for example, STRING, RESTRING, TIME, or any other
complex(5) type
■ Relational operator used by the scheduler
■ Requestable flag, which determines whether users can request the attribute for a
job
■ Consumable flag, which identifies the attribute as a consumable resource ■ Default request value that is taken into account for consumable attributes if jobs
do not explicitly specify a request for the attribute
■ Urgency value, which determines job priorities on a per resource basis
2.1.4.10.2 Defining Consumable Resources To enable consumable resource management, you must define the total capacity of a resource. You can define resource capacity globally for the cluster, for specified hosts, and for specified queues. These categories can supersede each other in the given order. Thus a host can restrict availability of a global resource, and a queue can restrict host resources and global resources.
The consumption of the resource is then monitored by Grid Engine software internal bookkeeping. Jobs are dispatched only if the internal bookkeeping indicates that sufficient consumable resources are available.
You define resource capacities by using the complex_values attribute in the queue and host configurations. The complex_values definition of the global host specifies global cluster consumable settings.
To each consumable attribute in a complex_values list, a value is assigned that denotes the maximum available amount for that resource. The internal bookkeeping subtracts from this total the assumed resource consumption by all running jobs as expressed through the jobs' resource requests.
2.1.4.10.3 Multiplied Resource Requests Versus Non-Multiplied Resource Requests By default Grid Engine performs multiplied resource requests, which means that a consumable resource request is multiplied by the number of slots allocated to a parallel job. The configuration for multiplied resource requests is designated by a YES flag in the consumable column of the job row in the complex definition.
The following multiplied resource request is explained below:
qsub -l mem=100M -pe make=8
Grid Engine multiples the consumable resource request (100 M) by the number of slots allocated for the parallel job (8). The consumable usage is split across the queues and hosts on which the job runs. If four tasks run on host A and four tasks run on host B, the job consumes 400 Mbytes on each host.
While multiplied resource requests typically work well, in the case of software licenses, it is more practical to make a per job request, or a non-multiplied resource
request, which debits the exact amount requested. Starting in Grid Engine 6.2u2, you can configure the complex to accept non-multiplied resource requests by changing the jobs consumable flag from YES to JOB, as shown below:
#name shortcut type relop requestable consumable default urgency #--- jobs j INT <= YES JOB 0 0
2.1.4.10.4 Examples - Defining Consumable Resources Only numeric attributes can be configured as consumables. Numeric attributes are attributes whose type is INT, DOUBLE, MEMORY, or TIME.
In the QMON Main Control window, click the Complex Configuration button. The Complex Configuration dialog box appears.
To enable the consumable management for an attribute, set the Consumable flag for the attribute in the complex configuration.
To set up other consumable resources, follow these examples: