This module is responsible for testing service compositions and their con- stituent services. Test execution is performed online i.e., in parallel to the normal operation of the service composition. As advocated by our approach, online testing of service compositions is used to complement runtime monitor- ing. The data collected by the Online Testing Module is similar to the data collected by the Runtime Monitoring Module.
For performing online testing activities, the Online Testing Module has two components, the Service Composition Tester for testing service compositions, and the Service Tester for testing constituent services.
8.3.1
Service Composition Tester
For coverage assessment, our approach uses (online) testing execution traces in combination with runtime monitoring traces. In this view, online testing can be triggered in cases when coverage of the service composition falls below a predefined threshold. The assessment of coverage can be performed using ap- propriate coverage criteria for service compositions. In Chapter 6, we proposed coverage criteria for service compositions.
For testing a service composition, test cases need to be selected. In this version of PROSA, we assume the test cases are available and stored in a Test Case Repository. The selection of test cases depends on the coverage criteria used for the service composition. The following are techniques for the coverage criteria presented in Chapter 6:
• The Intra-plan-local Coverage Criterion (see Section 6.1.2): The test case used for covering the target abstract service using the target execu- tion plan will be selected in case the execution trace of the test case is invalid.
• The Inter-plan-local Coverage (see Section 6.1.2): The test cases used to cover the abstract service using all execution plans will be selected in case the execution traces of these test cases are invalid.
• The Intra-plan-global Coverage Criterion (see Section 6.1.3): The test cases used to cover the service composition using the target execution plan will be selected in case the execution traces of these test cases are invalid.
• The Inter-plan-global Coverage Criterion (see Section 6.1.3): The test cases used to cover the service composition using all execution plans will be selected in case the execution traces of the test cases are invalid. For executing the selected test cases on the service composition, the Service Composition Tester sends the input defined in the test cases to the service composition execution engine, which then executes the service composition. The results of online testing (i.e., execution traces) are collected by the Service Composition Monitor in the same manner of collecting execution traces from normal usage of the service composition.
Using the Service Composition Monitor to collect both types of traces (i.e., test and monitoring) requires distinguishing the collected traces as either com- ing from monitoring normal system usage or from online testing. Technically, this can be achieved by attaching a tag when sending a request for testing purposes. The tag identifying the source of request is stored along with the execution trace. The default value for the tag will be identifying a monitoring
request. In case of testing, the value will be set for identifying a testing request as such.
For test case selection, it is necessary to know which execution trace be- longs to which test case. As we defined execution traces, each execution trace includes input. Using the combination of input and testing activity identifier (i.e., the tag), we can identify test cases by looking for execution traces of particular input and identified as testing.
8.3.2
Service Tester
In addition to collecting execution traces for the entire service composition, the Online Testing Module is responsible for actively collecting observations about the constituent services. This activity is performed by the Service Tester com- ponent which complements the activities performed by the Listener. Therefore, the Service Tester invokes the operations of the services to identify potential modifications in the behaviour or the performance of used services. To this end, the Service Tester invokes the services using the input defined in the test cases stored in the Test Case Repository. The Service Tester can be configured to invoke services at predefined rates. For example, it can be configured to invoke a service whenever the usage rate of the service drops below a certain threshold.
The test execution can be implemented using existing test execution envi- ronments. In earlier versions of PROSA [99], we have used an existing services monitoring framework, SALMon, for performing online testing of constituent services. SALMon is designed following SOA principles, which allows easier integration in other frameworks and allows deploying SALMon functionality as third-party services. Moreover, SALMon combines both testing and moni- toring in a single framework.
SALMon provides the Monitor Service (offered as a Web service) as the central access point for: (1) configuring monitoring and online testing for the different services in a service composition, and (2) retrieving Monitoring Data. The Monitor Service has a testing component which invokes the services using the provided test input and test rate. To collect Monitoring Data from monitoring or online testing, the Monitor Service creates one or more measure instruments which implement the logic required to compute concrete quality
metrics of the service.
Once the monitor is configured, it dynamically activates a concrete set of Measure Instruments depending on the required quality metrics to measure. Monitoring is performed through an Enterprise Service Bus (ESB). That is, instead of directly invoking the services, all requests and responses are sent through the ESB which in turn feeds the Measure Instruments.
The results of testing are observed and compared with expected results as defined in the test cases. The Dynamic Binding Information in the Data Repository is updated to reflect the observed modifications.