The SLO and compensation expressions are commonly specified by means of con- straints or functions of their related service metrics. Thus, a quite straightforward way to interpret the checking of the introduced properties (c.f. sections §A.3 and §A.4) is in terms of constraints satisfaction problems (CSPs). Similar techniques have been previ- ously applied by the authors to analyse: the conflicts between agreement terms [126],
A.5. MATERIALISING THE VALIDITY CHECKING
Mapping for Compensation Function
mapCF(SLOm, P, R)
(Pm[1].cond⇒Pc[1].value
Penalised AND ... AND
Region Pm[N].cond⇒Pc[N].value)
AND
Rewarded (Rm[1].cond⇒Rc[1].value
Region AND ... AND
Rm[N].cond⇒Rc[N].value)
Neutral AND
Region (NOT(Pm[1].cond OR ... OR Pm[N].cond
OR Rm[1].cond OR ... OR Rm[N].cond) ⇒c==0)
Table A.1: General Mapping from Compensation Function to Constraint
the SLA fulfilment at monitoring time [129], and the compliance between agreements and agreement templates [123]. CSPs are defined as a set of variables, their domains and a number of constraints, and there are a number of solvers that use satisfaction method to find a solution for the CSP, if it exists. Our approach involves mapping the iAgree SLA to a CSP and then using a CSP solver to check the validity properties. Therefore, next we thoroughly explain this mapping, which is summarized in Tables §A.1 and §A.2. In order to exemplify the CSP mapping we will use the SLA presented in Figure §A.8 (GNWT-4) as a running example in the rest of the section.
Mapping the Compensation Function (CF): As a common element included in every CSP used to analyse each property, we map the compensation function defined for each CG to a constraint. Table §A.1 describes this mapping. Each(value, condition)pair in penalties or rewards is mapped to a constraint of the form condition ⇒ variable == value. As neutral compensations are not usually explicit, an additional constraint, where compensation is zero outside the defined penalty and reward expressions, is added. Finally, all these constraints are combined with an AND operator. Note that this mapping corresponds to a well-defined piecewise function. A previous operation to check that the compensation is a mathematical well-defined function could be per- formed before compensation analysis.
Our running example (GNWT-4) includes a guarantee with a compensation func- tion that expresses a penalty when metric elapsedDaysPercent is greater or equal than 120, and a reward when the same metric elapsedDaysPercent is equal to 80. There- fore, the compensation definition for GNWT4 would be mapped to the next constraint:
mapCF(GNWT4) = { {(elapsedDaysPercent>=120 ⇒InvoicePercent==10)AND (elapsedDaysPercent==80 ⇒InvoicePercent== −5)AND !(elapsedDaysPercent==80 OR elapsedDaysPercent>=120) ⇒ (InvoicePercent==0)}
This mapping is used as a base to create the CSPs to check the validity of CF and CG, which are described next. In next CSPs, the metrics and the compensation variable are the CSP variables and their corresponding domains are the CSP domains.
CSP for Consistent Compensable Function (Property 1): Since CSPs can only solve sat- isfiability problems (i.e. exists a solution) and we have to check the accomplishment of this property for all possible values, we verify this property by transforming it into an existential constraint and check that the resulting CSP is not satisfiable. Furthermore, in this CSP we duplicate the metrics variables so that we can check if two different vari- ables with the same constraints (i.e.: same compensation function) have inconsistent compensations. For instance, if the CF is defined on metric elapsedDaysPercent, we define two different metrics, elapsedDaysPercent1 and elapsedDaysPercent2, with the same compensation function to check the consistency. The property is evaluated by adding a constraint that relates the two compensations and utilities for these vari- ables so that one variable can have both a higher compensation (i.e. higher penalty) and a higher utility than the other. If the resulting CSP is satisfiable, it implies that the function is not consistent. Otherwise, it would mean that there are two values of the metric variable for which one metric value has both higher penalty and higher utility than the other value.
Considering our example, where the utility function (U) is the reverse of the previ- ous metric, elapsedDaysPercent (and is inferred according previous subsection), as a greater delay in delivery is less useful for the customer (and the opposite), we get the following CSP: cspCCF(GNWT4)= { {elapsedDaysPercent1, elapsedDaysPercent2, InvoicePercent1, InvoicePercent2}, {int[−200..200], int[−200..200], int[−100..100], int[−100..100]},
{mapCF(GNWT4−1)AND mapCF(GNWT4−2)
AND(InvoicePercent1>InvoicePercent2)AND
A.5. MATERIALISING THE VALIDITY CHECKING
As this problem is satisfiable, the example compensation function is inconsistent. This result is an effect of the wrong compensation definition, which only rewards when the value for elapsedDaysPercent is exactly 80, but does not reward more useful val- ues as elapsedDaysPercent = 60.
CSP for Threshold Saturated Compensable Function (Property 2): For this property, the generation of CSP receives two additional input values, the saturation thresholds, so we only check if the metrics are outside these two values. In this case, we consider it not saturated. In our example, using 30 and -30 as input thresholds for saturation, the CSP for checking the saturation in the example compensation function is:
cspTSCF(GNWT4, 30,−30) = {
{elapsedDaysPercent, InvoicePercent},
{int[−200..200], int[−100..100]},
{mapCF(GNWT4)AND
(InvoicePercent>30 OR InvoicePercent< −30)}}
As the compensation value cannot reach maximum or minimum values in any case (its only possible values are 10, 0 and -5), the problem is not satisfiable and the com- pensation is therefore saturated.
CSP for Valid Compensable Function (Property 3): A compensation function is valid if it is consistent (Property 1) AND saturated (Property 2). In our example, the compensa- tion is invalid (it is saturated but not consistent).
CSP for Consistent Compensable Guarantee (Property 4): To check the consistency of compensable guarantees, we evaluate SLO accomplishment together with the compen- sation function. This property involves checking one existentially quantified CSP, there exists a value that is neutral and fulfilled, and one universally quantified CSP, no value is fulfilled and penalised or unfulfilled and rewarded at the same time. Therefore, we build two different CSPs. First, if there is a neutral compensation region (NCR) for an SLO fulfilled value, and then if there is a positive compensation value (penalty) when the SLO is fulfilled or a negative compensation value (reward) when SLO is not ful- filled (consistent compensation regions, CCR). In our example, the CSPs for checking the consistency between compensation function and SLO are:
cspNCR(GNWT4) =
{elapsedDaysPercent, InvoicePercent},
{int[−200..200], int[−100..100]},
{mapCF(GNWT4)AND(InvoicePercent==0
cspCCR(GNWT4)= mapCF(GNWT4)AND
{elapsedDaysPercent, InvoicePercent},
{int[−200..200], int[−100..100]},
{mapCF(GNWT4)AND(InvoicePercent>0
AND elapsedDaysPercent<120)OR
(InvoicePercent<0 AND !(elapsedDaysPercent<120)}
As cspNCR is satisfiable and cspCCR is not satisfiable for the elapsedDaysPercent metric, the compensable guarantee is consistent.
CSP for Valid Compensable Function (Property 5): If the compensable guarantee is consistent with compensation function (Property 4) and the compensation function is valid (Property 3). For our example, the compensation definition is invalid, hence the compensable guarantee is also invalid.
A.6
VALIDATION IN REAL-WORLD SCENARIOS
In this section we describe how we have validated our proposal. In particular, the goal of the validation was to answer the following research questions:
• RQ1: How expressive is our compensations model in comparison to real-world SLAs? We want to know whether the compensation model that we use is ex- pressive enough to model a wide variety of real-world SLAs and which are the characteristics of the SLAs that we are not able to express.
• RQ2: Are the compensations in real-world SLAs valid according to our notion of validity? The validity of a CG is at the central part of this paper, so we want to know whether CG of real-world SLAs follow this notion of validity.
• RQ3: Which difficulties appear when modelling SLAs defined in natural lan- guage?All real-world SLAs are expressed in natural language. Therefore, before checking their validity it is necessary to formalise them. With this question, we examine the problems that may appear in this step.
To answer these questions, we have modelled with iAgree and evaluated up to 319 compensable guarantees that are described in natural language in 24 different scenar- ios6belonging to three different domains, namely: cloud service providers (e.g., Ama- zon, Google, or Rackspace), non-cloud service providers (e.g., DHL, train companies or telecommunication services), and B2B service outsourcing.