NOTA: Utilizar una punta diferente para cada cepa, y evitar tocar los medios de cultivo con la punta de la micropipeta

To assess the attributes of interest of DivRep components in a single framework and handle the inherent uncertainty in the assessment we propose the use of “uncertainty explicit” methods. We have illustrated in the previous subsections how the assessment of the two attributes, performance and reliability, can be performed for ranking of DBMS products. The ranking can be used for selecting components of DivRep middleware - two servers that turn out to be the “best” according to the ranking would be selected. This is not ideal because it is possible that two servers, which scored best individually do not form the best pair, e.g. the servers’ performance might be different once the replication protocol is in place. It is worth noting that the described method is not intrinsically related to DivRep and it can be applied in a wider context where need for similar ranking of other products arises.

In the example of the assessment method (Section 5.4) we have used somewhat arbitrary definitions of incorrect response failures. A better alternative for more representative assessment of reliability attribute can be found in (Gashi, Popov et al. under review), where a set of faults (bugs) from four different database severs form a demand space, which is used for more stressful testing of the components under assessment. More interestingly, the results obtained from the related study also led to CS1 being the best server. This gives an extra confidence that the method indeed performs plausibly, i.e. does not lead to counterintuitive results, which would have required further scrutiny to explain why the perception of CS1 as the best is not supported by the results of the study.

Using the same approach Gashi et al. (Gashi and Popov 2007) show how an optimal pair of components can be selected to form 1-out-of-2 fault-tolerant system. The focus of the paper is on the assessment of the reliability and not performance. They use the same Bayesian model described here, applied to a dataset of faults reported for four database servers, to choose the pair with the lowest probability of the coincident failure. In this way the authors cater for the possibility that the best pair might not be built with the database servers which are the best individually (indeed they confirmed this possibility). However the approach is applied to running the servers on their own,

i.e. a reported fault is executed against an individual server. It would be problematic to apply the approach to performance only – an overhead due to the replication (various synchronisations) may lead to performance significantly different from when the servers are run on their own.In any case, theapproach can be combined with what is presented in this section with the aim to select the “best pair” taking into account both fault logs and performance.

The definition of the prior distribution is fundamental in Bayesian assessment. In our study we have assumed that prior distributions for each component are the same. This was due to the unavailability of other known ‘objective’ evidence that we could use to define more accurate priors. Anecdotal evidence about the servers does exist, but is difficult to translate these subjective beliefs into priors in the form required by our method. By assuming that the prior distributions were the same for each server, the decision on which server is chosen is dictated by the observations only. As a result the decision of the types of distributions for the random variables in our study becomes less important.

However there are other ways of deriving more accurate priors. We could, for example, utilize evidence from earlier versions of the servers and then do multiple steps of inference, i.e. if we want to perform the assessment with later versions of the servers in our study (e.g. with versions of PostgreSQL after release 7.2 or Firebird after release 1.0) we can use the posteriors derived here as priors for the later versions, collect the new evidence for the later versions and then use the model to derive the posteriors for each. This approach has also been reported elsewhere (Littlewood and Wright 1997).

The method of assessment proposed in this paper would be applicable to different families of COTS components. The setup described in Section 5.3 and illustrated in Section 5.4 is particularly relevant for COTS components with stringent reliability

and performance requirements. We provided empirical results using off-the-shelf database servers. Java Virtual Machines (JVMs), various application servers, web servers and Business process execution engines (Andrews, Curbera et al. 2003) are also examples of COTS components were reliability and performance requirements are usually the deciding attributes for selection. Fault and failure reports, which can be used as observations, do exist for these products and so do performance benchmarks (e.g. ab benchmarking tool for web servers (ApacheSoftwareFoundation 2008)). Therefore, similar measurements to what we did for database servers are also possible

with these other families of COTS components. In many cases for these components one may not need to deal with more than 2 attributes, which makes our 2-attribute model proposed in Section 4 immediately applicable without any further simplifications.

6. Related Work

Every extension of knowledge arises from making the conscious the unconscious.

Friedrich Nietzsche

Earlier sections discussed different topics related to the research and each of them contains the references to the relevant work. In this section, we describe in detail the work related to a particular topic, database replication, in order to emphasize it as one of the central themes in the research work.