Internal validity
Internal validity is the degree to which a measured and observed effect among research variables can be said to be due to a causal relationship (Fellows and Liu, 2015). Internal validity is the extent to which the relational and causal effects, which the researcher observes between the research variables, may be the correct evidence for the conclusions (Creswell and Creswell, 2017).
Also, Creswell (2014b) described internal validity as the rate at which a researcher can draw correct conclusions that there are causes and relational effects among variables, which might be influenced by attributes of the participants, and maturity and biases in the selection process. Internal validity is also referred to as content validity, which examines whether variables are representative of possible items, and criterion-related or construct validity, which examines whether scores relate to an external standard or measure as intended. This can be done by means of statistical procedures, or consulting external experts (Creswell and Clark, 2017).
In this study, the consideration for internal validity focused on the quality of and relation between scores obtained from the results of the questionnaire survey, preventing potential biases in the expert estimations, and how the quality of the questionnaire responses and expert estimations
108
influenced the quality of findings and conclusions. It becomes imperative to consider content, criterion-related and construct validity, because the main concern of questionnaire survey designs with regard to internal validity has to do with the quality of the scores obtained from the questionnaire.
Content validity refers to the extent to which questionnaire items are representative of all facets of the constructs being measured. The current research ensured content validity through an in-depth review of extant literature and adopting the impact matrix (ISO, 2018) from which the questionnaire items (probability of occurrence and surety of events) were derived. Criterion- related validity describes whether the obtained sources link to some external standard, such as the score on a similar instrument and construct validity measures what they are intended to measure (Holt and Goulding, 2014). This validity was addressed by employing universal linguistic scales of measurement for the questionnaire items, to collect valid scores from the respondents.
According to Fellows and Liu (2015), the regression analysis and Spearman Rho correlation coefficient are two instruments to evaluate the criterion-related and construct validity. In this study, the Spearman correlation between two parts of the questionnaire survey (part 2.1: the probability of occurrence and part 2.2: degree of severity of events) for both cost and time was calculated and is shown in Tables 4.7 and 4.8.
According to Tables 4.3 and 4.4, positive and significant correlation was found between part 2.1 and part 2.2 of both costs (r=0.81; p<0.01) and time (r=0.74; p<0.01).
Table 4.7: Spearman correlation between Part 2.1 and 2.2 of cost questionnaire
Part 2.1 Part 2.2 Part 2.1 Correlation Coefficient 1.000 0.81** Sig. (2-tailed) . 0.000 N 32 32 Part 2.2 Correlation Coefficient 0.81** 1.000 Sig. (2-tailed) 0.000 . N 32 32
**. Correlation is significant at the 0.01 level (2-tailed).
Table 4.8: Spearman correlation between Part 2.1 and 2.2 of time questionnaire
Part 2.1 Part 2.2 Part 2.1 Correlation Coefficient 1.000 0.74** Sig. (2-tailed) . 0.000 N 32 32 Part 2.2 Correlation Coefficient 0.74** 1.000 Sig. (2-tailed) 0.000 . N 32 32
109
In order to further evaluate the quality and relation among of scores obtained from the results of the questionnaire survey, the researcher took into account biases that could lead to errors in the estimation of probabilities. Biases are caused by the incorrect use of heuristics or rules of thumb by experts during estimating (Cantarelli et al., 2013). Biases in expert estimations are outlined in detail in the literature in Chapter Two.
The most common biases encountered in the estimation process (Hallowell and Gambatese, 2010) were:
Availability - the experts tend to base their estimates on the ease, which enables them to retrieve the information from their short-term memory; Anchoring - the experts arrange their estimate of a probability to an initial value and adjust, but with an inadequate adjustment; Contrast - the expert’s perception of a variable was influenced (enhanced/diminished) by the exposure to a (larger/lower) value of the immediately preceding variable; Overconfidence - experts tend to give narrower confidence intervals compared to real intervals.
The optimism bias did not apply to the experts in this specific study, since none of them was a planner or a promoter of the case study project. Biases were avoided with calibration by experts and careful preparation of the estimation protocols. Calibration methods involved instructing the experts on the correct use of estimation values, probability concepts, and biases. Care was also taken to, inform the experts about the use of their estimation’s results in the uncertainty model to estimate total construction cost and time of the case study project; to improve the knowledge of the experts with the advanced probability distributions and correlations in the construction of projects by discussing and visualising patterns; to warn the experts about the existence of availability and anchoring biases in the process of estimation, and their effects on the estimation results; to discuss the overconfidence bias when estimating the lowest and the highest values or small and large correlations, to prevent the experts from committing the same type of error; and to prevent the contrast bias by randomisation of the sequence of the activities’ estimations in the estimation protocol.
Additionally, the estimation exercised the outside view, by not allowing the experts to focus on the details of the project, but instead, it provided estimation and opinion from extensive experience.
External validity
External validity relates to the degree to which the results of the survey can be generalised to the population (population validity) and possibly other research settings or contexts (ecological validity) (Yin, 2017). External validity is essential to quantitative research using some standardised procedures in selecting a sample as representative of the population study (Creswell and Creswell, 2017).
Since this study used a nonbiased population and employed standard procedures for selecting an appropriate population (professional construction managers of highway projects under construction in 2017 in South Africa, with more than 20 years' experience) and used a census survey (data collection from the whole population), automatically the population validity of
110
external validity was fully satisfied, and no further test was required. Although there was no wrong or right answer to any question in the questionnaire, the ecological validity was ensured as well.
Reliability
Reliability in quantitative research measures the internal consistency of the data collected from the respondents. It is assessed by the statistical reliability coefficient (Creswell and Clark, 2017). Cronbach's alpha is the most common statistical test to check the internal consistency of a research variable (Field, 2013). Cronbach's alpha determines the degree of internal consistency or average correlation of items in a survey instrument from 0 to 1: the closer to 1, the greater the internal consistency of the items in the scale (Tavakol and Dennick, 2011).
Tavakol and Dennick (2011) provide the following classification for Cronbach's alpha coefficient range shown in Table 4.9.
Table 4.9: Cronbach’s alpha coefficient range
Cronbach’s alpha coefficient range Internal consistency
α≥0.9 Excellent 0.9>α≥0.8 Good 0.8>α≥0.7 Acceptable 0.7>α≥0.6 Questionable 0.6>α≥0.5 Poor 0.5>α Unacceptable
The internal consistency of each group of disruptive events for both cost and time variables was estimated. Tables 4.10 and 4.11 summarise the results from Cronbach’s alpha test obtained for cost and time.
Table 4.10: Reliability value of disruptive events for the cost component
Uncertainty group Disruptive event Cronbach’s alpha
Economic
1. Fluctuation of prices of materials and/or equipment
2. Monopoly of material and/or equipment suppliers 3. Saturated market
4. Fluctuation in foreign exchange rate
0.927
Environmental
1. Weather 2. Natural disasters 3. Remote location cost
4. Terrain (or topographical site)
0.872
Financial
1. Tax and/or legal fees 2. Cash flow difficulties 3. Poor financial control 4. Lack of capital 5. High tender price
6. High cost of materials and/or equipment 7. High cost of labour
0.777
Legal 1. Right of way acquisition
111
3. Difficulties in importing equipment and materials 4. Changes in government regulations and laws 5. Unclear arbitration process for legal disputes
between construction parties
6. Changing of bankers’ policies for loans 7. Ineffective delay penalties
8. Type of contract
9. Problem in dispute settlement due to law 10. Contract failure Political 1. Political situation 2. Encroachment problems 3. Human-made disaster 0.926 Social
1. Cultural heritage issue
2. Personal conflicts among labour 3. Social and cultural impacts 4. Rehabilitation of affected people 5. Disease 6. Security 7. Corruption 0.869 Technical General 1. Size of contract 2. Health and safety
3. Change order (change in the scope of the project) 4. Difficulty of schedule
5. Inadequate planning and scheduling 6. Payment delay
7. Contractual claim
8. Improper construction methods 9. Specification change
10. Poor communication/coordination between construction parties
11. Latent ground conditions
0.773
0.707
Labour
1. Inadequate labour productivity 2. Absenteeism of labour 3. Shortage of skilled workers 4. Poor quality of workmanship
0.712
Material
1. Unreliable supplier of material 2. Delay in material supply 3. Bad quality of materials 4. Shortage of materials
0.794
Equipment
1. Low efficiency of equipment 2. Slow mobilisation of equipment 3. Late delivery of equipment 4. Availability of equipment
0.876
Technology 1. Obsolete technology
2. New technology adoption 0.814
Consultant
1. Lack of experience in design and supervision 2. Inaccurate investigation of construction site 3. Frequent design changes
4. Incomplete drawings, specifications 5. Mistakes in design and/or specifications 6. Inaccurate time and cost estimation 7. Inadequate monitoring and supervision 8. Delay in decision-making
9. Lack of technical staff
0.732
112
2. Incorrect planning and scheduling 3. Frequent change of subcontractors 4. Poor quality of project management 5. Re-work due to contractor errors 6. Lack of technical staff
7. Incompetent contractor/subcontractor
Overall Total: 76 events 0.759
Table 4.11: Reliability value of disruptive events for the time component
Uncertainty group Disruptive event Cronbach’s alpha
Economic
1. Fluctuation of prices of materials and/or equipment 2. Monopoly of material and/or equipment suppliers 3. Saturated market
4. Fluctuation in foreign exchange rate
0.914
Environmental
1. Weather 2. Natural disasters 3. Remote location cost
4. Terrain (or topographical site)
0.851
Financial
1. Tax and/or legal fees 2. Cash flow difficulties 3. Poor financial control 4. Lack of capital 5. High tender price
6. High cost of materials and/or equipment 7. High cost of labour
0.753
Legal
1. Right of way acquisition 2. Deficient documentation
3. Difficulties in importing equipment and materials 4. Changes in government regulations and laws 5. Unclear arbitration process for legal disputes
between construction parties
6. Changing of bankers’ policies for loans 7. Ineffective delay penalties
8. Type of contract
9. Problem in dispute settlement due to law 10. Contract failure 0.718 Political 1. Political situation 2. Encroachment problems 3. Human-made disaster 0.901 Social
1. Cultural heritage issue
2. Personal conflicts among labour 3. Social and cultural impacts 4. Rehabilitation of affected people 5. Disease 6. Security 7. Corruption 0.875 Technical General 1. Size of contract 2. Health and safety
3. Change order (change in the scope of the project) 4. Difficulty of schedule
5. Inadequate planning and scheduling 6. Payment delay
7. Contractual claim
8. Improper construction methods
113 9. Specification change
10. Poor communication/coordination between construction parties
11. Latent ground conditions
Labour
1. Inadequate labour productivity 2. Absenteeism of labour 3. Shortage of skilled workers 4. Poor quality of workmanship
0.704
Material
1. Unreliable supplier of material 2. Delay in material supply 3. Bad quality of materials 4. Shortage of materials
0.756
Equipment
1. Low efficiency of equipment 2. Slow mobilisation of equipment 3. Late delivery of equipment 4. Availability of equipment
0.865
Technology 1. Obsolete technology
2. New technology adoption 0.802
Consultant
1. Lack of experience in design and supervision 2. Inaccurate investigation of construction site 3. Frequent design changes
4. Incomplete drawings, specifications 5. Mistakes in design and/or specifications 6. Inaccurate time and cost estimation 7. Inadequate monitoring and supervision 8. Delay in decisions making
9. Lack of technical staff
0.741
Contractor
1. Lack of experience in the line of work 2. Incorrect planning and scheduling 3. Frequent change of subcontractors 4. Poor quality of project management 5. Re-work due to contractor errors 6. Lack of technical staff
7. Incompetent contractor/subcontractor
0.822
Overall Total: 76 events 0.735
As presented in Tables 4.10 and 4.11, the results of the reliability test indicate acceptable to excellent internal consistency for both cost and time components with reliability coefficients ranging from 0.7 to 0.9.