Questionnaire design is influenced by the choice of survey method (Frazer & Lawley 2000). Being a mail survey, it is self-administered and difficult for subjects to clarify any doubts that they might have; and so the ‘Survey of Employees’ Workplace Experiences’ questionnaire should feature simple and straightforward. “Quality data require a well-designed study using a carefully crafted questionnaire” (Totten, Panacek & Price 1999, p. 26). Thus, a number of design issues including question content, question type, question wording, response format, scales and scaling, and structure and layout were duly addressed to minimize biases (Frazer & Lawley 2000;
Mangione 1995; Sekaran 2003; Totten, Panacek & Price 1999; Trochim & Donnelly 2007). The first three issues are discussed right below and the remaining three in the later subsections.
Question content, question type and question wording
The content and purpose of each question were carefully considered so that each construct was adequately measured (Frazer & Lawley 2000; Sekaran 2003). As each of the ten study constructs is of a subjective nature (e.g., satisfaction, commitment), where subjects’ beliefs, perceptions, and attitudes are to be measured, multiple closed questions were used to tap the dimensions and elements of each construct.
Closed questions are ideal for tapping subjective feelings for they help subjects make quick decisions to choose among several alternatives. Also, they allow for easy
coding of information for subsequent analysis. For the demographic variables (i.e., objective facts) such as age and gender, each of which was tapped by a single direct question. As regards wordings, simple English without slang, jargon or idioms was used in anticipation of subjects’ differences in educational levels and cultures.
In addition, when developing the survey questionnaire, the first step is to thoroughly search the relevant literature and look for previously validated instruments that can be adapted to measure the study constructs (Totten, Panacek &
Price 1999). Of the ten study constructs, the search results successfully identified validated instruments for all except the intention-to-return construct. The three subsections that follow describe the development of new instrument measuring intention-to-return, the adaptation of previously validated instruments measuring the other nine study constructs, and the questionnaire items measuring demographic characteristics.
Development of new instrument measuring intention-to-return
As the intention-to-return construct is of a subjective nature which cannot be measured directly, a multi-item scale instrument was developed to adequately tap the construct. The instrument development process involved several steps. First, ten (10) items considered having face validity (as in Appendix 1.2) were generated to tap into the intention-to-return domain. Most of these items were critically examined and adapted from the organisational climate questionnaire by Duxbury & Higgins (1999).
This initial step has been supported by two theoretical views that follow.
Forehand & Gilmer (1964) define organisational climate as the set of relatively enduring characteristics describing an organisation, which distinguishes the organisation from other organisations, and influences the behaviour of people in the organisation. For Litwin & Stringer (1968), organisational climate is the set of measurable properties of an organisation, perceived directly or indirectly by its people, which influences motivation and behaviour resulting in consequences such as satisfaction, productivity or performance, and retention or turnover. These theories provide justification for adapting measurement items from organisational climate questionnaire to tap the construct definition of intention-to-return.
Second, the set of ten (10) items as in Appendix 1.2 was submitted to the Supervisors of the study for evaluation in order to attest the content validity of the instrument. This step resulted in items 3 through 10 being replaced with two new items as in Appendix 2-10. Third, the resulted 4-item instrument was tested on a sample of 160 subjects participated in the pilot study discussed in the later subsection. In essence, the reliability and construct validity of the 4-item intention-to-return scale were established on the basis of desired levels of item loadings and internal consistency reliability, and desired evidence for convergent and discriminant validity.
Adaptation of previously validated instruments measuring study constructs
The nine previously validated multi-item instruments adapted for measuring the study constructs are summarized in Table 4.1 below, with details in Appendices 2-1 through 2-9. While these instruments using five-point Likert scale had their Cronbach’s Alphas ranging from 0.72 to 0.96, they were re-tested for reliability, construct validity, and wording appropriateness in the pilot study discussed in the later subsection.
Table 4.1: Summary of Previously Validated Multi-item Instruments
Questionnaire items measuring demographic characteristics
Previous studies of trust antecedents and outcomes have offered inconsistent views concerning the potential effects of respondent demographic variables on respondents’
scores. In particular, most focal empirical studies reported in chapter 2 have not considered respondent demographic variables as control variables in their model estimates. Further, for the previous studies that employed Mayer, Davis &
Schoorman’s (1995) trust model, Mayer & Davis (1999) used age, gender and tenure as control variables which yielded statistically insignificant effects on all the regression models. Also, Tan & Tan (2000) employed age, education, tenure and employment level as control variables which were found to be statistically insignificant. Again, for Davis et al. (2000) and Mayer & Gavin (2005), respondent demographic variables were completely omitted in their studies.
For the present study, items measuring demographic characteristics included age, gender, education level, job type, basis of employment, level of employment, and tenure in organisation as outlined in Appendix 2-11. While these demographic factors were not thought to have significant effects on the respondents’ scores for the reasons aforesaid, a pre-test for their statistical significance in the regressions specified for hypothesis testing was considered and further discussed in chapter 5.
In brief, each latent construct was tapped using a multi-item scale instrument, whereas each demographic variable was measured by a single direct question. In any case, short and clear closed questions were used in conjunction with plain and simple English. Associated with these question design decisions were the response format (also covers scales and scaling) and the structure and layout as discussed next.
Response format, structure and layout
“Careful attention to response format will save hours of data entry. People tend not to read directions, so using the same format throughout is preferable” (Totten, Panacek
& Price 1999, p. 33).
The response format for all multi-item scale instruments employed a six-point Likert-type scale with the following anchors: strongly disagree (=1), disagree, slightly disagree, slightly agree, agree, and strongly agree (=6). The six-point scale that leaves out the midpoint choice was used to minimize central tendency bias (Mangione 1995; Si & Cullen 1998). Also, several questionnaire items were negatively phrased and reverse scored in an effort to reduce acquiescence bias (Mangione 1995). Both biases are further discussed later in this subsection.
For the demographic data, all measurement items using category scales were placed in the last section of the questionnaire. This ‘placing demographic questions last’ decision has been supported by the predominant opinion of previous researchers (e.g., Frazer & Lawley 2000; Grinnell 1997; Mangione 1995; Sekaran 2003; Totten, Panacek & Price 1999). These researchers have generally agreed that demographic questions are boring, and ‘placing them first’ may also cause respondents to think that the researchers are more interested in their personal information than the survey objectives leading to respondent bias and respondents’ refusal in participation.
Overall, all questions were neatly aligned and logically organised in appropriate sections with clear instructions on how to complete them. A page entitled
‘optional respondent comments’ was also provided at the end of the questionnaire.
The two versions of questionnaire, one for the pilot study and the other for the main study, are shown in Appendix 2-12, and Appendix 3-12A, respectively. The main- study questionnaire contains 56 questions (49 scale items and 7 demographic questions) which require 8 – 10 minutes to complete. This optimal completion time arising from the questionnaire design efforts aimed at minimizing respondent effort that could improve response rates (Totten, Panacek & Price 1999). The questionnaire design efforts also attempted to minimize response set biases as discussed next.
Minimize response set biases
When deciding on the response format above-mentioned, due considerations were exercised to minimize response set biases namely acquiescence bias, beginning–
ending list bias, recall bias and central tendency bias as advised by Mangione (1995).
Firstly, acquiescence bias, the tendency to say ‘yes’ or ‘agreeable’, was dealt with by having some negatively phrased questions, and more scale points that made respondents to consider the fine points of their attitudes (Mangione 1995). Secondly, beginning–ending list bias, the tendency to pick items at the beginning or end of long lists (because people seldom read the whole list or they remember the items listed last), was minimized by having shorter lists of choices in the demographic section.
Thirdly, recall bias is the tendency to misremember particular information due to long recall periods used in questionnaire items. This bias did not cause any concern as all questionnaire items referred to recent experiences or current information.
Finally, central tendency bias is “the tendency to answer in the middle, to look average” (Mangione 1995, p. 34). On the use of odd-point scales, Mangione argues that “if you give people a middle choice they will use it” (p. 13). To minimize such a bias, he suggests using an even-point scale that leaves out the midpoint choice. Additionally, Si & Cullen’s (1998) study further confirms that the use of even-point scales does reduce central tendency bias, particularly in Asian cultures namely China, Japan and Hong Kong. Consistent with these researchers’ views, the present study employed an even-point Likert-type scale to minimize this kind of bias.
More about Likert-type scales follow.
Use of Likert-type scales
Likert-type scales are designed to examine how strongly subjects agree or disagree with duly constructed statements, ranging from most positive to most negative attitudes or feelings toward some object (Sekaran 2003; Zikmund 2003). They are commonly used to measure a wide variety of latent constructs in social science research (Kent 2001), as well as, in marketing research (Zikmund 2003).
In the present research, Likert-type scales were used to measure each latent construct for a number of reasons. First, they communicate interval scale properties to subjects, and hence produce data that can be assumed interval scaled (Madsen 1989; Schertzer & Kernan 1985; Sekaran 2003). Second, they were used in most previous studies (discussed in chapter 2) and the nine previously validated instruments above-mentioned, in which they were treated as interval scales. Finally, they are popular means for measuring attitudes because they are simple to administer, particularly they allow for easy categorization and coding of data for subsequent analysis (Totten, Panacek & Price 1999; Zikmund 2003).
When planning the use of Likert-type scales, due considerations were given to the issue of odd- versus even-point scales, and the impacts of number of scale points on reliability and validity. For the former, an even-point scale was used to minimize central tendency bias according to Mangione’s (1995) and Si & Cullen’s (1998) recommendations aforesaid. Here, one key decisive factor was that the multi-cultures of Western Australia and Singapore, particularly the Asians, are more prone to such a bias. As regards the latter, previous empirical studies have revealed that there is no optimal number of points in a Likert-type scale, for its reliability and validity are independent of the number of scale points (Jacoby & Matell 1971;
Steinberg 1990). For Mangione (1995), the use of six-point scales is adequate to help respondents make fine distinctions when responding to a complex and emotional issue, and hence reduces bias. This position is consistent with numerous studies employing six-point Likert-type scales (e.g., Babin, Boles & Robin 2000; Bernal, Wooley & Schensul 1997; Chang 1994; Hills & Argyle 2002; Misener & Cox 2001;
Niemi-Murola et al. 2007; Pomini et al. 1996; Skinner et al. 1991; Weist et al. 2005).
These considerations together, therefore, called for the use of six-point Likert-type scale in the present research.
To sum up, the survey questionnaire design followed certain principles of question content, question type, question wording, response format, scales and scaling, and structure and layout in order to minimize nonresponses and response set biases. Further, the questionnaire design, particularly the desired response format, also considered the need for easy categorization and coding of data for subsequent analysis. Once data were collected, the ‘goodness of data’ was then assessed through
tests of validity and reliability of the measures (Sekaran 2003) discussed in the subsection that follows.