The purpose of a survey instrument is to collect the information required to address the overall research question and hypotheses. In designing the survey, a set of questions must be developed to capture this information as well as to encourage respondent involvement and minimise response error (Malhotra 2004). The phrasing of questions needs to consider the characteristics of the respondent group particularly in terms of their education level. Lack of understanding of the question’s intent is directly associated with inaccurate or nonresponse (Malhotra 2004). Further, survey designers should be cognisant of other sources of response error; that is, the fallibility of the respondents’ memory, their motivation to
Chapter Three: Methodology and Method respond as well as their level of knowledge (Sudman and Bradburn 1982). Unfortunately there is not a scientific approach to survey design that will guarantee the optimal outcome, however, several guidelines exist (Churchill 2001; Malhotra 2004; Sudman and Bradburn 1982). These guidelines relate to question design and format, response design, question order, presentation of instructions, survey format and pretesting. These areas will be discussed as follows.
First, it is suggested that questions should be clear in terms of using simple and appropriate language, setting manageable tasks and supplying the required information (Converse and Presser 1986). Specific questions are preferable to general ones and both double-barrelled and leading questions should be avoided as they contribute to response error (Fink 2003). Questions should also be relevant and related to each other in a meaningful way. Negative phrasing should be avoided as respondents may inadvertently miss the negative qualifier if reading quickly (Babbie 1990; Punch 1998). Finally, it is suggested that where possible questions should be adapted from questions that have been successfully used in other surveys providing the context is appropriate (Fink 2003; Fowler 1993; Sudman and Bradburn 1982). This is consistent with the argument that the wording of a question is critical to maximising survey validity (Sudman and Bradburn 1982, Malhotra 2004).
With regard to question format, researchers must also decide between open-ended and close-ended questions. Open-ended questions allow for a greater variety of response that might otherwise be missed, but conversely they also allow for responses that may not match the intent of the question. Open-ended questions
Chapter Three: Methodology and Method are also more labourious with regard to data entry. Close-ended questions are easier to process but need to be constructed to provide all possible responses; this may require the inclusion of a category such as ‘other’. Further, the stated responses should ideally be mutually exclusive (Babbie 1990).
Another issue to be considered is whether a middle alternative should be provided in response to bipolar questions; that is questions in which respondents are asked to choose between two opposite positions. The reason for eliminating this option of indifference is that it forces respondents to take a stand in a particular direction. An opposing view suggests that respondents should not be forced to take a position, and that the middle-of-the-road position provides valuable information as to the intensity of the respondent’s view (Converse and Presser 1986). In terms of listing responses, it is suggested that the least socially desirable response be listed first to increase the likelihood that the respondent will read all possible answers.
Question order must also be considered to reduce any potential biasing effect; unfortunately there is no clear guidance as to when an order effect will occur (Sudman and Bradburn 1982). Researchers need to be aware that answers to some questions may have implications for subsequent questions in terms of providing a context, either intended or unintended, in which they will be answered. Researchers should be aware that survey validity may suffer from fatigue effects when respondents are required to address long lists of items or questions. In general, surveys should commence with the easiest questions and progress to those that may require more thought (Bourque and Fielder 2003).
Chapter Three: Methodology and Method Similarly, questions should also be presented in a logical order with general questions preceding specific ones. With regard to questions seeking demographic information, there is some dispute as to whether they are best placed at the beginning or the end of the survey (Bourque and Fielder 2003). The reasons for placing these questions at the end are because they may be perceived by the respondent as either boring or perhaps personal. In either instance, the upfront placement of these questions may impact on the likelihood of completion. On the other hand, an argument can be made for commencing the survey with these questions. That is, the questions are easy for the respondent to answer and the likelihood that they will be answered is maximised.
Due to the lack of interviewer presence, specific strategies have been recommended for developing self-administered questionnaires that focus on maximising user-friendliness and minimising error (Bourque and Fielder 2003; Fowler 1993). In general, it is essential that questions are unambiguous and detailed instructions are given (Malhotra 2004). Close-ended questions are particularly recommended and researchers should strive to format questions so they can be answered in the same way. Further, questions should be as short as possible and the instances where a respondent may be required to skip a question should be minimised.
In general, the instructions that accompany a survey are another important influence on the accuracy of the information gathered and are critical in the case of self-administered surveys. Instructions contained within a self-administered survey can be divided into three categories: general instructions, transitional
Chapter Three: Methodology and Method instructions and question-answering instructions (Bourque and Fielder 2003). General instructions include an introduction to the survey, a statement relating to the researcher’s interest in the respondent’s attitudes, experiences and opinions as well as instructions as to what the respondent should do with the completed survey. Transitional instructions advise the respondent that the question topic is changing or provide a context within which a question should be answered. Finally, question-answering instructions provide respondents information as to how to answer the questions, for example by circling the appropriate response.
The format of the survey also impacts on both the accuracy of response and completion rate. The format should assist the respondent in moving through the document. In general, it is suggested that questionnaires should be between four and twelve pages with adequate spacing between questions using 12 point type in an easy to read font (Bourque and Fielder 2003). To maximise legibility, italics should be avoided and it is preferable to use bold or capitals if emphasis is required. Questions should never be split between pages and response codes should clearly correspond to the relevant alternative. The survey should end with instructions as to what to do with the completed questionnaire and the respondent should be thanked for their time and effort.
Prior to administering the survey, it should be reviewed by experts as well as by potential respondents to allow the instrument to be fine-tuned (Fink 2003; Fowler 1993). It is particularly important to pre-test self-administered surveys as there is no interviewer to clarify what is required of the respondent. It is suggested that the best way to pre-test in this circumstance is by having a group of respondents
Chapter Three: Methodology and Method fill out the survey and then conducting a discussion with the respondents regarding clarity of instructions and questions as well as comprehension. Finally, a pre-test will also indicate the time required to complete the questionnaire. Researchers must strike a balance between collecting all of the information they require and ensuring that the accuracy and completeness of responses do not suffer as a result of respondent fatigue.
The specific design considerations for the surveys used in this research study will be described in section 3.4.4. The surveys are provided in Appendix 5.