Analysis of the data that will be acquired by the interviews will be an essential part of the study to gain an intrinsic understanding of the processes within the clarification phase. For the analysis of the data two main steps are used to process the data. First a within-case analysis will be used to report and structure the findings for the case at hand. The within-case analysis will help structuring the significant amount of data that is available after the interviews (Eisenhardt, 1989:a, Dul & Hak, 2008). The second step relates directly to the within-case analysis but is concerned with the cross-case search for patterns and generalizations of findings to draw overall conclusions. The theoretical framework will play a role in the analysis since it is the framework for analysis to understand the actual implementation of the clarification phase from two perspectives (Eisenhardt, 1989:a, Yin, 2013, Dul & Hak, 2008). The within-case and the cross-case analysis method will be introduced below.
Within-case analysis
Within-case analysis is the in-depth exploration of a single case as it is a standalone entity (Mill, Durepos & Wiebe, 2010). The within-case is considered to be an important first step in understanding and describing the phenomenon in the case study. Even in a multiple case study each case can contribute information to the findings of the entire study. Analysis of the individual cases is thus of interest for the entire study because of both the uniqueness of individual cases and possible commonalities between them (Ayres, Kavanaugh & Knafl, 2003, Eisenhardt, 1989:a, Mill, Durepos & Wiebe, 2010). The within-case analysis will help in the first step towards understanding how the process of the clarification phase within each case was filled in. The in-depth understanding that is created with this analysis can lead to the preliminary conclusions based on the findings in the individual cases. These findings and the results of the within-case analysis are important inputs for the cross-case analysis to generalize the findings were possible. The within-case analysis is initially used to develop a stand-alone understanding, conclusions, and recommendation of each case on which the cross-case comparison to identify commonalities is based, but also to understand unique elements in each case (Eisenhardt, 1989:a, Mills, Durepos & Wiebe, 2010). There are no definitive guidelines or approaches about how to conduct a within-case analysis (Mills, Durepos & Wiebe, 2010). For the within-case analysis in this study several techniques are used such as word tables to analyze the individual cases and structure the findings to create comparable data sets for the cross-case analysis.
Cross-case analysis and patterns
A multiple case study enables the use of cross-case analysis techniques to find patterns in the data. The cross-case analysis is useful to find patterns and generalize findings. The cross-case analysis is closely related to the within-case analysis (Eisenhardt, 1989:a, Yin, 2013). According to Miles and Huberman enhances the combination of within and across analysis of cases the accuracy of the evidence, establishes the generality of the facts and findings, is usable to clarify the relevance of particular findings, helps to test theoretical notions, and generate theory (Miles & Huberman, 1994). Overall the analysis is likely to generate more robust findings compared to a single case study only, or when conclusions are drawn from within-case analysis only (Yin, 2013). The structural analysis during the cross-case evaluation emphasizes to search further beyond the initial conclusions and impressions. The contextual influences on the various cases are reduced by generalization of the
Envisioned results
The analysis of case documents and especially the results of the interviews are envisioned to result in a good insight in the processes of the clarification phases of the various cases. Based on the various analysis techniques is it expected that an overview of conclusions and recommendations can be presented as a result of the case studies. Especially the recommendations to improve the efficiency and enable project teams to better utilize and plan this period will be valuable input for the development of a model which represents the improvements for the phase. The methods presented above are schematically presented in Figure 5.4 to indicate the process of the case studies.
findings above the individual level. During the analysis differences and correlations are appointed to evaluate the processes that have taken place during the clarification phase of the various projects and have leaded to certain projects and delays compared to the proposed planning. For the analysis of the cases various methods will be used to gain an intrinsic understanding of the data gathered and develop sufficient correlations to draw conclusions regarding the individual cases and across the various cases. Methods that are used for both the within-case analysis and the cross-case analysis will be briefly introduced below.
Explanation building
Explanation building is a technique for the analysis of data from case studies to develop explanations about the occurrences in a certain case (Yin, 2013). An explanation in a case study is intended to act as an answer to a specific research question (Mill, Durepos & Wiebe, 2010). In this study this relates to the intrinsic understanding how the process of the clarification phase is filled in and what the reasons are for the structural exceeding of the planning. To explain something means here “to contribute to fostering an understanding of it” (Mill, Durepos & Wiebe, 2010 p.368). Something is here seen as the clarification phase with it envisioned products and the corresponding process to produce them. The explanations add to the understanding of the phenomena observed in the case study. The cases will be used to generate an understanding of certain causality between the observed overrun of the planning and reasons why this occurred or not; the explanation. Contrastive explanations can be formulated based on the contrasting theoretical framework which adds to the exploration of rival explanations (Mill, Durepos & Wiebe, 2010, Yin, 2013). To a certain degree the expectations are shaped by the initial exploration of the implementation of the clarification phase through the analysis of evaluations.
Time-series analysis
Time-series analysis is a technique that can be used to understand the match between the observed and a rival trend. In theory these rival trends are observed against theoretical patterns or rival explanation (Mill, Durepos & Wiebe, 2010, Yin, 2013). During this study the time- series analysis will be operationalized in another manner; they are used to understand the proposed pattern (the planning) against the realized planning (or time-series) and were a significant part of the interviews. In Attachment G the overview of the planning per case is included as a tool to let the interviewees explain the process and to gain insight in important moments in which deviations occurred (main input from the interviewees is in these planning as well).
Cross-case synthesis –word table
Yin describes cross-case synthesis as a type of analysis for multiple case study designs and can be seen as an interpretation of the cross-case analysis. A specific method proposed by Yin is the use of word tables to analyze the available data from the individual cases. Word tables use one or more uniform categories to order and display data from various cases. The tables can be used for a qualitative analysis and utilized to draw cross-case conclusions about categories at hand. The use of word tables for the cross-case analysis is relying strong on argumentative interpretations (Yin, 2013). An empty word table is depicted in Figure 5.3.
Figure 5.3: empty word table (adapted from Yin, 2013).
Case study 1 ... Case study 4
Interview 1 Interview 2 ... Interview 11 Interview 12 Data topic x
Within-case analysis Cross-case analysis
Figure 5.4: overview process case studies.
CASE STUDY
SELECTION INTERVIEWS TRANSCRIPTS WORD TABLES VERIFICATION WITH INTERVIEWEES CONCLUSIONS PER CASE CONCLUSIONS RECOMMEN- DATIONS CONCLUSIONS THEORETICAL FRAMEWORK ANALYSIS CASE DOCUMENTATION WITHIN-CASE
ANALYSIS CROSS-CASE ANALYSIS
TIME-SERIES
ANALYSIS procedure
interviews structuring of case study data possible adaptations including important within case conclusions and recommendations
Checklist of client actions
Detailed project schedule
Checklist of client actions
5.7 RESULTS
In this section the results of the case studies will be presented. First the results of the project documentation will be presented briefly followed
by the results and analysis of the interviews.
PROJECT DOCUMENTATION
Project documentation has been analyzed to develop insight in both the tender specifications of the client and the products of the vendor as a background for the interviews. As a result of this analysis the products of the vendor in the clarification phase are aligned against the envisioned products to gain insight in whether the products of the clarification phase support the envisioned process of the Best Value
approach.
The products are considered from the perspective of dominant information and if they are detailed on the aspects which are perceived as key for the execution (e.g. elaboration of the top risks for the project goals, quality management system of the vendor and the alignment with the client). Dominant information must be interpreted from the perspective of Best Value and means that in the products the topic is presented so simple, apparent, relevant and is so important that is can predict the final outcome and leads to the same conclusions by everyone (Kashiwagi, 2013:b). Level of detail is additionally included since not all parts of the products are relevant to detail to a large extend. These used scores are as following:
• -2: Not included in the products.
• -1: Too limited included in the products (not dominant or demanded aspects are missing).
• 0: Meets to a high degree the envisioned products of the clarification phase.
• 1: Concept of dominant information is not leading and/or high levels of detail on topics which do not demand it.
• 2: Concept of dominant information is not leading and/or high levels of detail on many topics which do not demand it.
In Figure 5.5 the results of this analysis is presented in diagrams. As can be seen in the diagrams it is observed that there are notable differences in the products per case. Some of the notable results of the analysis are presented per case below.
2
Weekly Risk Report Milestone schedule Risk Management Plan Project Management Plan Project scope
Checklist of client actions Value added plan
Detailed project schedule
Project financial summary
Key Performance Indicators 1
0 -1
-2
2
Weekly Risk Report Milestone schedule Risk Management Plan Project Management Plan Project scope
Checklist of client actions Value added plan
Detailed project schedule
Project financial summary
Key Performance Indicators 1
0 -1
-2
Project Management Plan Value added plan
Milestone schedule Risk Management Plan Checklist of client actions
Detailed project schedule
-1 -2 Project scope
Key Performance Indicators 2
1 0
Weekly Risk Report Project financial summary
2
Weekly Risk Report Project scope
Project financial summary
Key Performance Indicators 1
0
Milestone schedule Risk Management Plan Checklist of client actions
Detailed project schedule
-1 -2
Project Management Plan Value added plan
CASE 3 CASE 1
CASE 4 CASE 2
Case 1
• The project financial summary is not clearly included in the products.
• The project management plan and the risk management plan are very extensive and detailed.
• The checklist of the actions of the client is not included clearly in the documents. The meetings and checklists of the client are too
implicitly included in texts and are not dominantly presented in the products.
• The project schedule is very extensive and detailed for the entire process. A detailed planning is even included for the actual realization
of the project (during the interviews the client interviewees indicate that the level of detail of the planning exceeds the envisioned
necessity).
Case 2
• The checklist of the client only includes a scheme of meetings in which the client is expected to be present. Only implicitly are some
indications made when something is expected during the process of the execution.
• The weekly risk report is not included in any product and not described how it is going to be included in the process of the execution
phase.
• The key performance indicators are missing entirely from the products of the clarification phase.
• The project schedule is very detailed.
• The risk management plan is elaborated for every risk that is signaled in this phase.
Case 4
• The project management plan is very detailed and excessive. It contains a very detailed elaboration of the requirements and includes
many additional elements that are not requested directly or contain so much detail that the outline is missing (e.g. information management is elaborated extensively).
• The risk management plan is very extensive and detailed for all client and vendor risks identified during this phase.
Case 3
• The risk management file misses risks that are introduced in the participation plan. The risks that are included in the risk management
plan (all other risks) are elaborate and very detailed.
• The weekly risk report is not introduced and specified when for example this process is started or how it is incorporated in the process of
the execution.
• The value added plan is not elaborated.
Conclusion
It is concluded that in general the project management plan and the risk management plan tend to be very detailed and go far beyond the requirement of products on outlines by the client. Regarding the project management plan is it concluded that vendors have difficulties to include the actions of the client in a comprehensive manner in the products since in project management plans for traditional project this is often not a requirement. The quality management system of the vendor has to be adapted towards the demands of the client. This is an intensive process which often demands high levels of detail to sufficiently include these processes in the project management plan. Regarding the risk management plan can it be concluded that there is no focus on the top risks which threaten the project goals but instead all risks are detailed during the clarification phase regardless of the risk score. This means that in the various cases hundreds of risks are detailed. For both the project management plan and the risk management plan can it thus be concluded that there is a lacking focus on key aspects which require details which leads to high levels of detail in (almost) entire products.
Notable is that some products that are envisioned by the Best Value approach are missing entirely in the products of some cases. An example is the Key Performance Indicators which can be seen as an important product of the clarification phase for the execution of the project. Also the incorporation of the weekly risk report in the products of the clarification phase (e.g. project management plan and link with the risk management plan) is a new tool which is specific for Best Value execution phase and is not included in all cases. This may be the result that the vendors do not have any experience with this tool or the execution of a Best Value project which can be a reason some products are missing, not included sufficient, or included with difficulty in the products.