Terremoto 16 de abril de 2016: Pedernales, caso “Comunidad La Chorrera” 45

Based on our research goal and the corresponding research questions presented in Section 5.1, we structure our research in software engineering undertaken in this thesis into the following four Software Engineering Work Packages (SEWPs):

• _{SEWP1:Software Engineering in Computational Science} • _{SEWP2:Hierarchical Integration of Domain-Specific Languages} • _SEWP3:DSLsfor a Marine Ecosystem Model

5.2. Research Plan

• SEWP4:Evaluation

For each work package, we give a short description of the research con- ducted as part of this thesis. In particular, we highlight which research questions are answered using which methods.

5.2.1

SEWP1: Software Engineering in Computational

Science

The first work package consists of identifying the specific requirements of scientists for a software engineering approach for computational science. For this purpose, in Chapter 6, we carry out a literature review to identify key characteristics of scientific software development and to explain why state-of-the-art software engineering techniques are poorly adopted in computational science. Based on this review, we are able to demonstrate thatMDSEapproaches are a promising starting point for adapting software engineering approaches for computational science.

Via the methods of literature review and argumentation, the work pack- age SEWP1 answers the research questions SERQ1 (What is specific about software engineering in computational science?) with its sub-question SERQ1.1 (Which software engineering methods are well-suited for being adapted for compu- tational science?).

5.2.2

SEWP2: Hierarchical Integration of Domain-Specific

Languages

Our second work package covers designing the Sprat Approach, which is anMDSEapproach specifically tailored for scientists from different dis- ciplines who collaborate to implement complex simulation software. By analyzing the software architecture of scientific simulation software (partly by studying literature, partly via argumentation), we find that such software can typically be or actually is constructed using a hierarchically layered architecture. To take advantage of the fact that the boundaries of the layers in such an architecture usually coincide with divisions between scientific disciplines, we integrate multipleDSLsinto aDSLhierarchy, which forms the core concept of the Sprat Approach. We employ the method of formal spec- ification—utilizing a combination of the Unified Modeling Language (UML)

(Object Management Group 2015c) with Object-Z (Smith 2000)—to define the notion ofDSLhierarchies without any ambiguities. Furthermore, we introduce a notation forDSLhierarchies and model the process of applying Sprat with theUML.

This work package—the results of which can be found in Chapter 7— addresses the research question SERQ2 (How can multipleDSLsbe integrated for scientific software development and how can they interact with each other?).

5.2.3

SEWP3: DSLs for a Marine Ecosystem Model

The third work package is concerned with the process of selecting suitable DSLsfor applying the Sprat Approach and the construction of a concreteDSL hierarchy. We present guidelines for choosingDSLsfor scientific software development projects and list key requirements for such languages. To illustrate theDSLselection process, we describe its application to our case study example of implementing the Sprat Marine Ecosystem Model with the Sprat Approach (see SEWP4). For those domains for which we could not identify suitableDSLsthat already exist, we designed new ones; namely, the SpratPDE DSLfor mesh-basedPDEsolvers and the Sprat EcosystemDSL for ecosystem simulation specification. We specify the meta-models of both of these languages using, again, a combination of theUMLand Object-Z.

The results of this work package, which can be found in Chapter 8, answer the research question SERQ3 (WhichDSLsare suitable for the imple- mentation of the Sprat Marine Ecosystem Model with the Sprat Approach and how can they be integrated into aDSLhierarchy?).

5.2.4

SEWP4: Evaluation

Our fourth work package addresses the evaluation of the Sprat Approach. Koziolek (2008) distinguishes three different evaluation types, which exhibit increasing degrees of external validity:

• _{Type I (Feasibility): In this form of evaluation, the authors of a method} apply it by themselves to an example of their choice. Thereby, this type of evaluation tests whether the method possesses certain desired properties under laboratory conditions.

5.2. Research Plan

• Type II (Practicability): This form of evaluation assesses the method to be evaluated when it is used by the targeted developers instead of its authors. In this way, Type II evaluations allow to test whether the method is actually suitable for the intended users.

• Type III (Cost-Benefit Analysis): A Type III evaluation consists in a full cost-benefit analysis of the method to be assessed. For example, applying the Sprat Approach potentially requires to design newDSLs, which leads to higher up front costs for a scientific software development project. We claim that these costs are more than compensated by the increased productivity of the scientific software developers and by the improved code quality achieved by employingDSLs. To test this claim, in a Type III evaluation, the same software development project is conducted at least twice with two different teams of computational scientists, one team using Sprat and the other not using it. Comparing the respective costs of both projects over their whole life cycle makes it possible to quantify the business value of the method to be evaluated. Since such studies are costly and difficult to conduct (e. g., one has to accurately control confounding variables such as developer expertise etc.), they are rarely put into practice.

To evaluate the Sprat Approach, we employ a mixed-method exploratory case study (Runeson et al. 2012) that combines Type I and Type II evaluation elements (for the reasons given above, a Type III evaluation is beyond the scope of this Ph.D. thesis). Our case study focuses on the engineering of the Sprat Marine Ecosystem Model and theDSLsdesigned in this process (see Chapter 9). The fact that, in the context of this thesis, we were able to use the Sprat Approach to implement a model as complex as the Sprat Model is, in itself, a demonstration of the feasibility of the Sprat Approach—i. e., a positive Type I evaluation of the approach as a whole. Throughout this thesis, however, we focus only on the evaluation of theDSLswe designed for implementing the Sprat Model, namely the SpratPDE DSLand the Sprat EcosystemDSL. By evaluating these languages, we also assess the Sprat Ap- proach itself because the increase in productivity and code quality promised by the approach can mainly be attributed to the quality characteristics of the hierarchically integratedDSLs.

To evaluate the SpratPDE DSL, we use micro- and macro-benchmarks for performance evaluation (Type I) as well as expert interviews with both

domain experts and professionalDSLdevelopers (Type II evaluation). For the assessment of the Sprat EcosystemDSL, we conduct an online survey among domain experts (Type II evaluation) that contains embedded controlled experiments and feedback questionnaires. Furthermore, we qualitatively compare the Sprat Approach and theDSLsused to engineer the Sprat Model to related research via argumentation.

The results of our mixed-method approach, which can be found in Chapters 13, 14, and 17, answer the research question SERQ4 (Does the Sprat Approach improve the productivity of scientists in developing software and does it raise the quality of solutions implemented by them?) with its sub-questions

SERQ4.1to 4.4. Note that one outcome of the research conducted in the context of work package SEWP1 is that the term quality of solutions in SERQ4 can be explicated in the following way: a solution is of high quality if it reconciles the conflicting quality requirements of performance, portability, and maintainability and if it produces scientifically reliable results. For further details on how the evaluation of the individualDSLsconfirms the effectiveness and efficiency of the Sprat Approach as a whole, refer to Section 18.1.4 of Chapter 18.

Chapter 6

Software Engineering in