PRACTICAS EN LABORATORIO

In the absence of published data, and in a small scale context, information must be collected by researchers. This can be done in the form of a survey administered either online, by telephone, by post or via interviews. In a survey context, important issues concern the nature of the information to be collected how questions are constructed and what characteristics are required of the questions and/or survey.

De Vaus (2002) considers that it is useful in the first instance to consider the type of information being sought in relation to behaviour, beliefs, knowledge, attitudes and attributes and that it is vital to determine whether these are involved in the survey as a whole or in particular questions.

112

Whittle (2012) states that further principles to consider in question design include reliability, validity, discrimination, response rate, absence of ambiguity and lastly relevance (De Vaus, 2002). Briefly, reliability exists if a question is answered in the same way at different times by the same respondent ceteris paribus. Validity requires that a question is actually well focused on the desired attribute. De Vaus, for instance, considers that the use of an IQ test to judge intelligence may in fact be judging class background. Discrimination refers to the degree of variability in permitted responses. For example, questions permitting only extremes could not, by definition produce variety within the answers. De Vaus (2005) considers the examples of income and gender. The gender question with two (m/f) alternatives will yield the correct variation. However an income question, if it were as extreme as “do you earn under or over £100,000” would (presumably) in a typical population provide little variance in response, with the majority of people choosing the

‘under’ option even though there are significant income differences between the ‘under

£100,000’ respondents (Whittle, 2012). Discriminating via scales or bands tends to allow a greater variance in the sample on the key variables and provides more information for analysis (De Vaus, 2005; Oppenheim, 1992).

The response rate is also of crucial importance in questionnaire design since non-response can be highly problematic in analysis, given that the cause of non-response may not be known. This problem can be minimised through consideration of appropriate question content, construction and length, for example whether to use multiple choice, Likert Scales, or clearly phrased open response questions. The time required for completion is also important. (De Vaus, 2005).

113

For analysis, it must be assumed that all respondents have answered the same question, that is, a question cannot have one meaning to one respondent and another meaning to another. Two respondents for example may differ in their definition of the word “rich”. To one respondent it may mean having significant savings, to another it may mean having a high level of disposable income. A recognition of this problem in the question design process will minimise the risk of double meaning (Oppenheim, 2002). Finally, De Vaus (2002) simply states that his last principle of question design is relevance, that is, whether the question fulfils a particular function in the survey.

As mentioned previously, reliability and validity are two key factors in question design (Whittle, 2012; Pallant, 2007; De Vaus 2005; Singer et al, 2004). Firstly, the reliability of a measure provides an indication of how free the measure is from random error. There are two main methods for this judgment, which are the temporal stability of the measure and its internal consistency (Whittle, 2012; Pallant, 2007; De Vaus 2005; Singer et al, 2004). The temporal stability of the measure is assessed by administering it to the same individual at different times and calculating the correlation. High correlations indicate a greater degree of reliability.

However, the nature of the variable being measured must also be taken into account. For instance, current income security could feasibly change within a short period of time and a low correlation may not be an indicator of poor reliability in the context of highly volatile economic conditions. Measures of stable characteristics should, however, generate a high correlation (Pallant, 2007).

114

Validity in measurement simply refers to a survey question “actually measuring what we think it does” (De Vaus, 2005, p96). Whilst there is no cast iron rule for the assessment of validity, Pallant (2007) suggests collecting extra data on the measures to determine content validity, criterion validity and construct validity. Construct validity is the aspect of validity described earlier within De Vaus’ principles of question design. Content validity is the level to which the measure samples from the intended domain of content, in other words, the level to which the object is measured by a question. For instance, querying the level of satisfaction within a free trade cooperative and excluding the possibility of saying “no” does not provide the complete scope of the subject. Criterion validity judges the chosen or new measure against existing measures in the field. If there is a high correlation in the answers, the measure is judged as valid.

Messick (1975) argues that proving validity of a survey is futile given the difficulties in proving measures within a specific construct. Instead, it is proposed that validity is situation-specific requiring not the validity of the survey itself to be justified, but the validity of the survey in specific situations. A final alternative measure of validity is face validity.

This is “the appropriateness, sensibility or the relevance of the test and its items as they appear to the person answering the test [survey]” (Holden, 2010, p637) and takes account of the opinions of those taking the survey and their interpretation of the questions.

Fundamentally measures of validity have many drawbacks, not the least of which is that they are often compared to existing theories or methods, which may not in fact be appropriate in the specific context of a particular research project.

115