Distribuciones asociadas al modelo normal est´andar

A great part of the work presented in this thesis relates to tools and ar- tifacts that have been built to solve the problems of scheduling dataflow applications. This kind of a research problem fits well with design-science, which describes a methodology for research where the target is to create new innovative artifacts for a specific problem domain, and, while this method

has not strictly been followed, it is relevant to compare the presented work with the main points of this research method.

According to March and Smith (1995) [91], design-science has two ba- sic activities, build and evaluate, and four types of products or artifacts, constructs, models, methods, and instantiations.

The build activity corresponds to building tools and methods for solv- ing a specific problem, and the build activity is performed to construct a prototype that shows that the proposed idea is feasible and enables a evalu- ation of the proposed approach. What separate design-science research from engineering is that the construction does not follow what is best praxis in the field, but instead, the purpose of building an artifact is to create new knowledge and to enable an evaluation of a new idea. It is therefore impor- tant in design-science that research resources are not used to build artifacts which are based on already known and evaluated ideas. [91, 74]

A constructed artifact then needs to be evaluated in order to answer the question regarding if the artifact contributes to the progress of the field. The questions that needs to be asked are: How well does the artifact perform the task? How does it compare to previous work? To be able to evaluate the artifact, some criteria for success and metrics need to be defined, such that results are comparable. Of course, if no previous methods have been able to achieve the same goals, the feasibility of constructing the artifact is already a valid result. [91, 74]

The building process related to this thesis, includes extending a code generation tool chain with a set of analysis and transformation operations. To evaluate how well the artifacts work, some metrics that can be compared are: how fast the generated code is when run on different platforms, how usable the tool is regarding how large parts of a program can be transformed in to something more efficient, and finally, how usable the tool is for a developer, where usable means how difficult the tool is to use and how much time it requires. These metrics can then be compared to previous work, however, some criteria for success are also needed to define what can be regarded as success. In this case, it is not required that the usability improves, but instead, it is a trade-off with the other metrics.

Design-science research must produce an artifact, according to the defi- nitions of March and Smith, we have the four types of artifacts as follows.

The first one, Constructs, provides the language in which problems and solutions are defined and forms a vocabulary for the domain [91, 74]. According to March the evaluation of constructs typically involve complete- ness, simplicity, elegance, understandability, and ease of use. In the dataflow domain, which is rather mature, the constructs have been developed over several decades and for this reason adding new constructs can typically be avoided. In this work, existing constructs are used to keep the discussion simple and not introduce overlapping constructs; instead, the focus is on

other types of artifacts.

Modelsare built as a set of proposition or statements describing rela-

tionships between the constructs and are used to improve the understanding of the problem and solution and ties the approach to the real world problems. According to March the models are evaluated in terms of their fidelity with real world phenomena, completeness, level of detail, robustness, and internal consistency. In this work, several models are constructed which each high- lights one property of the dataflow program that is being analyzed. These methods are for this reason presented formally to demonstrate the proper- ties related to completeness and consistency, as an incomplete model in the context of code analysis and transformation, is worthless. The evaluation of the models is presented in Paper 4, and Chapter 3.

The methods are algorithms or guidelines that describe how a specific problem should be solved, or more applicable in the context of this the- sis, how to search the solution space. According to March the evaluation of methods mainly regards their efficiency, generality, and ease of use. In the work of this thesis, this relates mainly to strategies related to schedul- ing, how the information from the models is used to perform the actual scheduling. The evaluation of methods is mainly presented as case studies in Chapter 7 of this thesis.

Instantiationsare the actual tools constructed to demonstrate the ap-

proach and show that constructs, models, or methods can be implemented in a working system. The instantiations are the artifacts that link researchers to the real world and show how the artifacts react to it and how users react to the artifact. According to March the evaluation of instantiations relates to the efficiency of the artifact and what impact it has on the environment and users. For the tools produced within this work, this would mean that a number of developers should be able to use the tools and feel that it helps them achieve some of their goals.

Evaluation of the Work The research work behind this thesis was not

directly based on design science, but because of many similarities regarding the goals of the research and the kind of experiments needed, it is relevant to evaluate this work as a design science research problem. This can be done by analyzing the seven guidelines, regarding design science research, given by Hevner et al. in [65].

The first one, Guideline 1: Design as an Artifact, is obviously followed as described above according to the definitions of March and Smith. The second guideline, Guideline 2: Problem Relevance, states that these arti- facts should provide solutions to important and relevant business problems. In this context, this means that the problem that is solved has a value in industrial applications and that it aids to the development in the field. The relevance cannot be evaluated from the research work but instead from the

problem statement motivating the research. The relevance of the schedul- ing problem comes from the fact that, using dataflow in many industrial applications is avoided due to the lack of methods for generating efficient code.

Guideline 3: Design Evaluation. It is crucial that the resulting artifacts are evaluated with proper metrics and sufficient experimental data. Within this context, when the artifact is part of a design tool chain, the relevant properties are related both to the usability of the artifact from the point of view of the user of the artifact, but also regarding how the artifact affects the performance of the dataflow program which is transformed by the arti- fact. For the user, the question is how much manual labor and additional expertise is needed to use the artifact and how much time the user must wait for the tool to finish. These properties are evaluated in Chapter 7 which provides a set of case studies where the artifact is used to construct schedulers for composed actors. While several of the studied examples are automatically transformed by the artifact, the ones that are not are also im- portant for giving an improved understanding of the research problem. For the second part, regarding the performance of the actors that are composed by the artifact, the evaluation requires numerous experiments including dif- ferent configurations and target platforms. Experimental data is presented in Chapter 6 and in several of the original publication, however, what is even more important than being able to show promising numbers is that the experiments are properly performed. We will get back to evaluating the measurements shortly, after discussing the other guidelines by Hevner et al.

Guideline 4: Research Contributions. Following the discussion on de-

sign evaluation, a more general evaluation is defining clear and verifiable contributions of the work. According to Hevner et al. the contribution can be the design artifact itself or contributions related to foundations and methodologies if the area of the design artifact. While much of the work pre- sented in this thesis evaluate experimental strategies for building scheduling models and deriving schedules for composed actors, the design artifact both shows that such methods are implementable but also enables an evaluation of methods presented. The following guideline, Guideline 5: Research Rigor, relates to the research contributions by requesting rigorous methods in both the construction and evaluation of the design artifact [65]. In the context of this work, rigor relates to the set of dataflow programs that are used to design and evaluate the artifact such that the artifact can be shown to solve a general problem and not only a special case. Similarly, rigor can be related to the data sets that are used to perform experiments and evaluation of the artifact; rigorous experiments make the knowledge acquired from the exper- iments more general. Furthermore, to show that the presented methods are complete, with respect to the possible inputs to the artifact, the methods need to be described mathematically based on an formal description of the

dataflow program.

The next guideline, Guideline 6: Design as a Search Process, defines the research as an iterative process, where available means, which are re- sources available to construct a solution, are used to reach desired ends, representing goals or constraints on the solution, while satisfying laws in the problem environment [65]. For complex problems, the first version of an artifact typically requires simplifications of the problem space or decompo- sition in to subproblems. While such an artifact hardly can be expected to as such be usable, it enables an improved understanding of the problem and raises relevant questions. The last guideline, Guideline 7: Communication

of Research, elaborates this idea by highlighting the communication of the

research results to the relevant audience. The important result of the work is then not the artifact itself but the knowledge acquired from constructing the artifact and by conducting the experiments enabled by the artifact.