Métodos de control

First, a discussion was held about what requirements to involve. The participants were instructed to try and select those that they would consider the best for differentiating between alternatives for the subsequent ranking of alternatives. This resulted in seven main requirements that would be considered. Consequently, these requirements were ranked by the participants using pairwise comparisons as described. This resulted in the prioritisation of criteria shown in Figure 29. Besides the four quality attributes of reliability, availability, performance and security, there are three functional requirements present. The Device_Access criterium captures the need for support of direct connections between a client system and an external device. The Subsystem_Var and Feature_Var criteria are specified to show that alternatives should support variability or changes in both the kinds of external devices (subsystems) that are connected, as well as the features from these that are supported by the control system. It can be seen that the first two functional requirements make for a large portion of the overall weight of criteria. Please also note that a small weight for an individual criterium does not necessarily imply that it is of low overall importance to the project. This ranking merely shows the relative importance of the criteria.

One constraint to the overall solution was that its performance needed to meet a certain level. At the time of the case study, the participants could not immediately assign a measure and norm to this, but they indicated that in the real-world this could well be done. Performance is also present as criterium, since better performance is seen as always preferred. Therefore, it can be used to compare alternatives, with the constraint that it should at least be within a certain threshold.

Figure 29 – First Prioritisation of Criteria

Figure 30 - Adjusted Criteria

The first decision of Communication Mechanisms was then considered. Within the scope of the case study, the search for alternatives involved practitioners’ experience or previously used approaches. Some examples were also suggested by the researcher, but the ultimate selection was up to the participants. For communication mechanisms, two alternatives were deemed as possible candidates; a publish/subscribe (PubSub) or publish/asynchronous (PubAsync) response mechanism. These two alternatives were then ranked using pairwise comparisons for each criterium, resulting in the ranking shown in Figure 31.

DECISION-MAKING IN A MICROSERVICE ARCHITECTURE PAGE 93/130

Figure 31 - First Ranking of Communication Mechanism Alternatives

When this outcome was shown to the participants, they indicated that they largely felt that the outcome fitted with their beliefs. However, upon seeing the differences in weight that now exist because of the large influence of the top two functional requirements, the participants felt that these were too influential. When for instance looking at the support for Device_Access between PubAsync and PubSub, the difference felt superficial to participants. They indicated that there could indeed be differences in the support for this requirement between the two alternatives, but that in practice they expected that they could make both alternatives support this function. They felt the same for the other Subsystem_Var and Feature_Var functional requirements; as long as these features were supported, the relative differences between alternatives with regards to these were not as important for the comparisons as the quality criteria. Because of this, the choice was made to omit the functional requirements from the comparisons and keep the requirements for support of these functions as constraints. The ranking of criteria and alternatives for communication mechanisms could then be recalculated. The results of this are shown in Figure 30 and Figure 32. The participants indicated that the new comparison between alternatives was better suited with their beliefs and gave them more insight. As can be seen, the publish/asynchronous response mechanism has the highest score in this case, though not by much. The practitioners were confident though that they should choose PubAsync for this decision and continue.

Figure 32 - Adjusted Ranking of Communication Mechanism Alternatives

The next decision to be made would normally be Service Interconnection. However, at this point in the case study, there was not much time left since the discussion on and adjustment of the requirements took quite some time. Participants indicated that they were highly interested in looking at the Service Discovery decision, as they were encountering it in their daily work at the time. The decision was therefore made to move on and focus on that decision. Two feasible alternatives as way of implementing service discovery were selected; either client-side or server-side discovery. The choice between these was not constrained by the

DECISION-MAKING IN A MICROSERVICE ARCHITECTURE PAGE 94/130

choice of communication mechanism. The same pairwise comparisons with regards to criteria were then made, resulting in the ranking shown in Figure 33.

Figure 33 - Ranking of Service Discovery Alternatives

In this case, a clear preference for server-side discovery is observed. Participants felt that this outcome fitted well with their views and scores given during comparisons.

Another step in the communication category is to consider interface design; which is shown as a guideline in the decision-making overview. The main guideline here is to use a standardised IDL whenever possible. Participants were asked whether it would make sense to include this in the overall decision-making process and whether the guideline is usable. They indicated that both were indeed the case.

The next step would then be to discuss the decision made. Putting the separate chosen alternatives together to form a coherent solution requires some discussion and a reality-check. With the limited decisions made in this case study, this discussion by the participants was quite short. They felt that the best scoring alternatives were a good fit for the system and were confident that they could also find a solution to the service interconnection challenge that fit with the others. The final step of evaluating the decision outcomes by putting the solutions into practice was obviously out of scope for this case study, but participants indicated that they felt this was a logical and necessary step in the process. Especially since they use an agile software development method, this step felt natural.

8.2.2

Results

After the execution of the case study, the participants were first asked to comment on how they felt about the use of the decision-making framework for making these decisions. The first and foremost remark made by most participants is that they liked the use of the pairwise comparisons by the AHP method a lot and were pleasantly surprised by it. They felt that by comparing requirements and alternatives in pairs, the discussion about why one is better than another is stimulated as opposed to having to rank all options by direct scoring. The fact that each participant could input their own comparisons and eventually consolidate all into a single overview was also seen as helpful. Participants recognised the possible sources of bias that could be present when this would be done in a plenary fashion. One software architect indicated that this could also be very useful as justification for the choices made for a certain system, as these are now made explicit and after analysis give good insights. Participants were also asked about whether they thought the overview and dependencies between challenges in the artifact were technically correct. They indicated that they could not find any apparent shortcomings in both.

Points of improvement that were mentioned at this point were mainly about tooling; this was seen as still too ad-hoc and of a quite technical nature. Participants indicated that it might help to make the entire process a bit more ‘hands-on’ by for example creating a planning-poker-

DECISION-MAKING IN A MICROSERVICE ARCHITECTURE PAGE 95/130

like approach to the pairwise comparisons or using post-its on a whiteboard to show pairwise rankings. Besides this, participants indicated that they would have liked a more practical description of what decisions to make and how to make them. Having a short and to the point ‘how-to’ guide would in their opinion help to make it easier to use for practitioners.

The most notable observation during this case study was the fact that the requirements selection and prioritisation needed to be changed during decision-making. This is contrary to the intention of prioritising requirements and only changing them in subsequent iterations of decision-making. When participants were asked about why they thought the requirement selection was off in the beginning, they indicated that they probably needed more information about how to select the most architecturally significant requirements. They also noted that after they became more familiar with the decision-making framework, they felt that they could better foresee how the requirements would be used later in the process. Thus, better explanation beforehand and practice with the framework could also aid in improving requirement selection and prioritisation. Furthermore, participants indicated that they had some trouble in deciding what weight to give in comparisons. The 1-9 scale used here was not completely clear; especially the question of when to give a score of 9. More guidance on this would expectedly also help them make better judgements according to the participants. In short, observations from practice can be summed up as follows:

- Allowing each participant to give their own rankings and then consolidating was seen as helpful to reduce biases present in collective decision-making;

- The outcome of the decision-making process can possibly be used to document and justify the decisions made;

- No shortcomings could be found in the overview of and dependencies between challenges;

- The requirements selection process in this case study was suboptimal, resulting in having to change the considered requirements during its execution;

- The tooling used during the case study was seen as too complicated and could be more ‘hands-on’;

- More guidance may be needed to explain the selection and ranking of requirements and alternatives;

- The use of AHP for comparisons was received well and thought to be useful for evoking discussions.

All four participants filled out the survey to assess usefulness, ease of use and decision quality. The scores for each question in Appendix D were recorded and then processed to show overviews of the scores for each indicator. No data was missing, and no outliers were removed. All answers were coded on a scale of -3 through 3. Corresponding answers are shown in Appendix D. The average score per question is shown by a bar, and the overall average for the indicator is shown on top of the diagram. It can be seen that the participants ranked the usefulness and usability of the artifact somewhat favourably. The decision quality indicator received a low score, though, meaning that participants on average neither agree nor disagree on the questions regarding this. It must be noted though that the scale used for measuring these questions was different from that for usefulness and usability, so these scores cannot directly be compared. These two were measured by asking about the likeliness that certain statements would apply to the respondent, whereas decision quality was measured by asking about the extent to which respondents agree with statements. The average score that participants gave about perceived practicality of the outcomes was -0,25,

DECISION-MAKING IN A MICROSERVICE ARCHITECTURE PAGE 96/130

though no technical shortcomings inhibiting practicality of the framework were found during the case study.

When asked about why participants gave the decision-quality and practicality related indicators low scores, the consensus was that they were hesitant to trust their judgements because during the process the requirements and their prioritisation had changed. Participants expected that when they were more comfortable with the use of the framework, and had more guidance in requirements selection and prioritisation, they expect to be more convinced that the choices made are the right ones.

Figure 34 - Survey Results Case Study 1