COMPAÑÍA GENERAL DE COMBUSTIBLES S.A

The two terms reliable and valid were mentioned to describe how the measure constructed in this chapter differed from the approach typically used by outreach activities to devise questionnaires. To have faith in the results reported it is important evaluation research uses standardised measures that are reliable and valid. Validity and reliability are two key concepts for the acceptance of a new measure; validity focuses on establishing the accuracy of the measure. Reliability is a measure of consistency; this is either within the scale (internal consistency) or across people’s responses at different moments in time (test-retest reliability). There were a number of ways to measure validity and reliability.

8.3.1VALIDITY

Before the application of a measure in a research project validity must be established (Picho, Katrichis and McCoach, 2010). There are several types of validity that should be considered in scale construction.

8.3.1.1 Face and content validity

Face and content validity are the two most basic methods of validation that require no statistical technique to measure. Face validity assesses whether the items appear to measure what they are intending to measure. For the scale in this chapter, this was established through using focus groups with the target respondents to construct the scale items then subsequently checking the items. This process ensured the items were relevant, clear and unambiguous. Thus the aim of the measure was clear to the respondents motivating them to respond (Kline, 2000).

106 Content validity is considered to be a more sophisticated version of face validity. To establish content validity, judgments by experts in the field of widening participation were sourced to ensure the measure was comprehensive and covered all the domains it intended to measure (Onwveglouzie, Bustamante and Nelson, 2010).

Face and content validity are prerequisites for the acceptance of a new measure (Streiner and Norman, 2008). Therefore much time was spent to ensure the items were clear, easy to follow, there were no biases in item content and all relevant issues were covered. This was because poorly designed items at this stage of the scale construction would have led to problems later on in the scale development (Worthington and Whittaker, 2006; Streiner and Norman, 2008).

8.3.2.2 Construct validity

Construct is described to be a ‘mini theory’ which can explain a relationship among various behaviours and attitudes (Streiner and Norman, 2008). Most psychological scales tap into some aspect of a hypothetical construct (Streiner and Norman, 2008). For instance, the concepts of motivation, attitudes and self-confidence are constructs that are not directly observable so are referred to as hypothetical constructs.

Construct validity is a more rigorous approach used in scale development to establish validity. The approach assesses the extent to which the measure is a good representation of the construct being evaluated. It is central to the appraisal of a measure. The inclusion of construct validity demonstrates the separation between non-scientific methods used at present to devise questionnaires to evaluate outreach activities and the scientific approach to be used to develop the measure in this chapter.

Cronbach and Meehl (1955) suggested that construct validity comprises of three steps; set out the theoretical constructs to be measured and how they relate, develop a scale to measure these hypothetical constructs and test the relationship between the constructs and their observable manifestations. The emphasis on theory is to focus thinking about the theoretical issues prior to the scale process to increase the chance of the scale impacting on the literature (Clark and Watson, 1995). The ideal method to establish construct validity is factor analysis, a technique used to validate newly constructed scales. Factor analysis is a technique to identify a smaller number of factors or latent constructs from a larger set of observed items

107 whilst retaining as much of the original information as possible (Worthington and Whittaker, 2006; Field, 2009).

Construct validity can also be assessed by investigating the relationship between other constructs, both related (convergent validity) and unrelated (discriminant validity) (Pallant, 2006). Establishing construct validity is an on-going process, there is no one single experiment to demonstrate construct validity as new predications can made be from learning something new about the construct tested (Striener and Norman, 2008)

The development of the scale in this chapter used the approaches discussed above to establish the validity of the measure. This ensured that the measure was measuring what it was intended to measure. Reliability of the scale is also of importance when developing a new measure.

8.3.2RELIABILITY

Reliability is a measure of consistency, by either people’s responses (external

reliability) or consistency (homogeneity) within the scale (internal reliability). The reliability of a measure can be assessed in two ways; internal reliability also known as internal consistency and test-retest reliability which can be referred to as external reliability. Measures of reliability are independent of each other.

Internal reliability also known as internal consistency, explores how the items hang together, ensuring the items are related in a similar way. It provides valuable information about the item homogeneity of a measure and is important in scale development. Internal consistency is usually measured using Cronbach’s alpha (α), which assesses within-scale item intercorrelation. Cronbach's alpha is recommended in comparison to Kuder-Richardson formula and spilt-half reliability to examine internal consistency, as Cronbach's alpha can be used with binary-type data and α is the mean of all possible spilt-half reliability solutions. Cronbach’s alpha should be above .7 for a measure to be considered to be of good internal reliability (Streiner and Norman, 2008). If internal consistency is low it is assumed the measure is measuring more than one variable (Kline, 2000). A limitation of assessing internal consistency is that the reliability will depend on the number of items in the scale and this was taken into consideration. Cronbach's alpha is the technique most commonly used and was used in this research project.

108 External reliability is more commonly known as test-retest reliability and explores whether the measure produces similar results with the same people, it is the extent to whether the scores on the measure do not change considerably over a short period ensuring the measure is reliable over time. Test-retest reliability involves the completion of the measure on two separate occasions and the two sets of scores are reported to be positively correlated. For a measure that is to be used for evaluative purposes, high test-retest reliability ensures that changes in scores in the intervention group are reliable and not due to an unstable measure where scores fluctuate under unchanging conditions. Thus can reliably report that changes in outcome measures used for evaluation purposes are potentially due to the intervention itself.

The measure was developed using the approaches discussed to establish the measure to be reliable and valid.