To ensure the reliability of this review internal and external threats to validity were considered during the design phase of this SLR.
Internal validity threats to an SLR include researcher bias and how well the review answers the research questions. Threats to internal validity were minimized by designing a protocol based on the guidelines outlined by Biolchini et al. [Bio+05], Kitchenham [Kit04], Kitchenham and Charters [KC07], Kitchenham et al. [Kit+04], Sjoberg et al. [Sjo+07], and Wohlin et al. [Woh+12]. However, this review was conducted by a single researcher, and even though steps were taken to eliminate bias, it cannot be completely ruled out. Opportunities for researcher bias may have been present when rejecting or classifying borderline papers.
The search strategy and the inclusion and exclusion criteria were designed during the planning phase, and so should minimize threats to internal validity. There is the potential that important papers on the area of complexity metrics for process models may not have been uncovered by this review, because these were present in research databases not included in this SLR. This threat was minimized by using a diverse set of seven research databases, and by using a backward snowballing approach to uncover papers not uncovered during the initial search.
External validity threats to an SLR include how applicable the findings of the review are to the research question. The findings of this review are consistent with those of Polančič and Cegnar [PC16] who conducted an SLR of complexity metrics for process models. Although, Polančič and Cegnar [PC16] only addressed one of the questions contained within this review (RQ1 above), it must be noted that their SLR focused on a ten years period (from 2005 to 1 February 2015), and found only 66 complexity metrics for process models. This review also identified the 66 metrics identified by Polančič and Cegnar as part of the 281 identified metrics. The purpose for conducting this SLR was to identify metrics and research methods that could be adapted to create and validate complexity metrics for CMMN. The findings of this review appear to have served its expected purpose, which was to offer a basis for identifying complexity metrics for CMMN, as well as to identify research methods to validate these metrics.
As described by Kitchenham and Charters [KC07] any conflict of interest on the part of the researcher should be disclosed. This researcher had no conflict of interest with this SLR, as stated in the application for ethical clearance from UNISA (see Appendix D file 15 (2016-05-23 MAMarin_Student_Ethical_Clearance-v5.pdf)).
6.6
Summary
This chapter contributes a current SLR of research into complexity metrics for process models to the general body of knowledge in the area of complexity metrics for process models. This review was designed to identify metrics and research methods that could be applied to CMMN. The review identified complexity metrics for process models featured in published research conducted over the last 20 years (from the beginning of 1996 to the middle of 2016), and how these were validated. The SLR followed the guidelines for software engineering literature reviews created by Biolchini et al. [Bio+05], Kitchenham [Kit04], Kitchenham and Charters [KC07], Kitchenham et al. [Kit+04], Sjoberg et al. [Sjo+07], and Wohlin et al. [Woh+12], and adapted the quality evaluation of rigor and relevance described by Bin Ali et al. [Bin+14], Dybå and Dingsøyr [DD08], and Vasconcellos et al. [Vas+17] for its purposes. The information gathered in this SLR forms the basis for Chapters 7 and 8.
Metrics for Case Management
This chapter analyzes the applicability of the process modeling complexity metrics that were identified in Chapter 6 for Case Management Model and Notation (CMMN) [OMG14a]. The chapter contributes a set of CMMN metrics and sub-metrics to the broader knowledge base and also provides the theoretical validation of the proposed metrics. An evaluation of the method complexity of the CMMN notation was conducted in Chapter 4, and it was concluded that it compares favorably against other methods. Expanding on the work done in Chapter 6, this chapter goes one step further and defines process modeling complexity metrics for CMMN. A Systematic Literature Review (SLR) was conducted in Chapter 6 that did not uncover any complexity metrics for declarative process models that could be directly applied to CMMN. This chapter fills this that gap in the literature by defining process modeling complexity metrics for CMMN. The formalization of metrics in this chapter is based on the CMMN formalization that was discussed in Section 3.3. The complexity metrics for CMMN proposed in this chapter will be empirically validated in Chapter 8. Material from this chapter was previously published in Marin et al. [Mar+15b].
This chapter is organized as follows. Section 7.1 uses the results from the SLR conducted in Chapter 6 to identify metrics that can be adapted to CMMN. Section 7.2 defines a set of proposed metrics for CMMN. Material from this section has been published in Marin et al. [Mar+15b]. Section 7.3 validates the proposed CMMN metrics using the formal framework for software measurements as defined by Briand et al. [Bri+96], and the properties for software complexity measures as defined by Weyuker [Wey88].
7.1
Applicability of Current Process Metrics to Case Manage-
ment Modeling and Notation
The SLR conducted in Chapter 6 did not uncover any complexity metrics for declarative process models or for data-centric process models that could be applied directly to CMMN. All of the notations for process models used in the primary and secondary papers in the review were for imperative process models (see Figure 6.9). The literature review identified 206 non-duplicated process modeling complexity metrics (see Table 6.4). It was possible that some of the metrics that were uncovered by the review could be adapted for use in CMMN. Table 55, see Appendix D in file 28 (SLR-analysis.pdf) present the analysis that was conducted during this review to identify metrics that could potentially be used as a basis for complexity metrics for declarative process models. An analysis of each metric was conducted in order to identify suitable metrics for CMMN (which is a declarative process model) where all of the metrics were analyzed and classified into 14 clusters that could be used to create metrics for declarative process models (see Table 7.1).
The analysis presented in Table 7.1 informs the CMMN metrics that are proposed in Sec- tion 7.2. However, not all of the suggested metrics in Table 7.1 were used because only metrics that were thought to have a good possibility of capturing the complexity of CMMN were proposed in Section 7.2 . Most of the applicable metrics were counters, followed by cognitive complexity metrics where a set of weights was used to calculate the metric. The SLR identified several cognitive complexity metrics including [Çoş14; GL06a; GL06b; SW03]. These cognitive complexity metrics are not directly applicable to CMMN because they are based on control structures [Fig+10] like sequence, branching, iterations, etc., which are common in imperative process models but not present in CMMN. However, weights could be given to other elements in the model, based on how complex those elements looked to an observer.
Table 7.1:Process metrics that could be adapted to CMMN
Cluster Metrics Potential suggestion
Case count NSP([Abr+10] see Table B.34) Count case tasks (type: Counter)
Collapsed stages NCS([Rol+06b] see Table B.9), TNCS ([Rol+06b] see Table B.9)
Count collapsed stages (type: Counter) Data count NDO([Abr+10] see Table B.34), TNDO
([Rol+06b] see Table B.9)
Count data objects (type: Counter)
Durfee square DSM([KN12] see Table B.37) Adapt Durfee square met- ric (type: Calculated)
Table 7.1 – Continued from previous page
Cluster Metrics Potential suggestion
Event count TNE([Abr+10] see Table B.34), SE ([Men07] see Table B.16), TNE ([Rol+06b] see Ta- ble B.9)
Count events (type: Counter)
Halstead D([Car+06] see Table B.10), N ([Car+06] see Table B.10), V ([Car+06] see Table B.10)
Adapt Halstead’s primi- tive measures (type: Cal- culated)
Hierarchy depth Depth([La +11b] see Table B.35), HH ([Kre10] see Table B.33)
Depth of stage hierarchy (type: Counter)
Hierarchy width WH([Kre10] see Table B.33) Width of stage hierarchy (type: Counter)
Modeling concepts DMC([La +11b] see Table B.35) Count modeling concepts used in the model (type: Counter)
Perfect square PSM([KN12] see Table B.37) Adapt perfect square metric (type: Calculated) Stage count TSAC([L¨15] see Table B.40) Count stages (type:
Counter) Task count NOA([Car+06] see Table B.10), NA ([Abr+10]
see Table B.34), TNT ([Abr+10] see Ta- ble B.34), NOBA ([Muk+10b] see Table B.30), NA([GL06b] see Table B.11), TBAC ([L¨15] see Table B.40), AC ([Car05b] see Table B.8), SN ([Men07] see Table B.16), TNA ([Rol+06b] see Table B.9), NT ([Rol+06b] see Table B.9), TNT([Rol+06b] see Table B.9), SizeA ([Ant+11] see Table B.36), NOA ([HB09] see Table B.28), NN ([Kre10] see Table B.33), NA ([Gar+03] see Table B.5)
Count tasks (type: Counter)
Unique tasks NCl([Kre10] see Table B.33) Count unique tasks (type: Counter)
Weights CCBP([Muk+10b] see Table B.30), CW ([GL06b] see Table B.11), CCYAWL ([GL06a] see
Table B.14), CADAC ([Çoş14] see Table B.39)
Assign weight to elements and sum them (type: Weighted)