Estrategia Didáctica Digital

Reynolds and Diamantopoulos (1998) maintain that there is wide agreement among marketing scholars that pre-testing is an integral part of the questionnaire development process. As Hunt et al. (1982, p.270) pointed out, the researcher needs to ask: “Will the instrument provide data of sufficient quality and quantity to satisfy the objectives of the research?”(p.270). The benefits of a pre-test prior to conducting the main survey have been supported by numerous researchers (see Hunt et al., 1982; Blair and Presser, 1992; Churchill, 1995; Reynolds and Diamantopoulos, 1998; Zikmund, 2003). Pre-test is defined as “a trial run with a group of respondents used to screen out problems in the instructions or design of a questionnaire” (Zikmund, 2003, p.229). This section starts with a discussion of pre-test methods and justifying the ones employed within this thesis. This is then followed by a discussion of the pre-test sampling frame and procedures.

Blair and Presser (1992) found real differences between pre-test methods. This was confirmed by Reynolds and Diamantopoulos (1998), who noted several disagreements among scholars about the best method for pre-test administration. Overall, the methodological literature has been found to distinguish between three types of pre-test methods (Hunt et al., 1982; Blair and Presser, 1992; Churchill, 1995; Reynolds and Diamantopoulos, 1998; Zikmund, 2003), including planned field survey, personal interviews (face-to-face), and expert panel. The first of these, planned field survey, employs a small sample referred to as ‘pre-testing’ (Zikmund, 2003). The second, personal interview is where the interviewer is required to identify any obstacles, difficulties, or incomprehensible questions blocking respondents’ ability to provide accurate answers. The third is when an expert panel is asked to judge the instrument and determine any problems it presents.

The above three methods are critically analysed by Reynolds and Diamantopoulos (1998), who found that a planned survey is useful because it covers all aspects of the field survey, and is less likely to be affected by interaction between the respondents and interviewer. However, a problem with this method is that respondents who are not the targeted sample might complete the questionnaire. Therefore, they suggest that personal interview is the most effective means of conducting a pre-test, due to the

accuracy and completeness of the information generated. Although this method is subject to errors resulting from interaction between the interviewer and participants (i.e., bias introduced by interviewers), expert panels (the last method) could be used to determine if there are problematic questionnaire items. In order to minimize any error or bias, all of these methods have been used (see pre-test procedures).

4.6.1 Pre-Test Sampling Frame

Hunt et al. (1982, p.269) posed two main questions in discussing the sampling frame for a pre-test. These questions were “who should be the subjects in the pre-test?” and “how large a sample is needed for the pre-test?” For the first question, it was necessary to include subjects who were similar to those approached in the actual survey (Churchill, 1995; Tull and Hawkins, 1990). Hence, a small number of respondents with certain characteristics were deemed to be more efficient in exploring errors in the survey instrument than respondents chosen randomly from the population of interest (Reynolds and Diamantopoulos, 1998).

The sampling frame for a pre-test consists of loyal guests of five-star hotel chains that correspond with the population to be studied. These subjects have formed the population of interest in the purposive sample generated from four selected hotel chains in Jordan (Dead Sea Movinpick, Petra Movinpick, Aqaba Movinpick, and Le Meridien hotels).

In the case of pre-test sampling size (the second question), there is little agreement in the literature (Hunt et al., 1982). For example, Zatalman and Burger (1975) did not specify size, simply recommending a ‘small’ sample. Others such as Boyed et al. (1977) indicated that a sample of 20 is adequate. Luckas et al. (2004) point out a size of 50 respondents allows the running of proper statistical testing procedures. Accordingly, 200 questionnaires were distributed to loyal guests at these hotels, aiming for a completion of at least 50 respondents.

4.6.2 Pre-Test Procedures

Because there are limitations to each of the pre-test methods, many researchers have recommended using different combinations of approaches (i.e., Blair and Presser, 1992; Malhotra, 1993; Churchill, 1995). As a result, expert panel, interviews, and planned field survey methods have all been used to pre-test the questionnaire of this thesis in order to overcome the shortcoming of using one method (see Table 4.8).

The first procedure involved distributing the draft to a panel of four experts. Two of them were professors in the area of marketing at Victoria University in Australia, and two others were professors in hospitality setting. One of them is at Victoria University in Australia whilst the other one is at Strathclyde University in Scotland. These four experts were asked to evaluate the questionnaire to: 1) assess the relevance of its conceptualization of marketing research operation; 2) appraise the suitability of the terminology to the hotel context; and 3) make further suggestions, criticism and comments on the questionnaire and its facets.

The second procedure was to ensure that this instrument could be used within the Arabic culture. The Arabic translated version was then presented to three Arabic experts. One of these was a professor in hospitality marketing area (Applied Science University in Jordan) and who speaks both languages fluently, while the other two were managers at five star hotels in Jordan (Marriott and Hyatt). All these experts were asked to evaluate the questionnaire. They identified two items (or questions) related to financial bonds and three related to structural bonds that needed to be reworded to better reflect the hotel context. In addition, a positively worded item was added to the loyalty scale as discussed in section 4.4.4. Further changes were also required for several demographic questions relating to the number of nights that guests had spent in each hotel, purpose of visit, and the guest’s nationality. The necessary revisions were made to the instrument to ensure its relevance to the domain of this thesis and to achieve face validity.

The third procedure followed Bowen and Shoemakers’ (1998) suggestion in which five personal interviews were conducted. Five hotel guests were interviewed after gaining approval from the hotels. The purpose of these interviews was to ask the

respondents to identify any problems in regard to the questionnaire format, wording or design, and to address any comments or suggestions they had. As a result of this procedure, it was suggested that providing more space between each group of questions within the same part would make the questionnaire easier to read. It was also identified that two out of five respondents did not understand the question related to their opinion on how they felt about the relationship they had with their service provider. The questionnaire was modified and refined before conducting the pre-test survey.

Table 4.8: Procedures Used in Pre-test

Procedures Target Reasons this Procedure

Used

1. Panel of experts

2. Panel of experts after translation process

3. Personal interviews

4. Planned survey (data collection)

Two professors in the area of marketing + two professors in hospitality area

One professor in hospitality marketing (Applied Science University in Jordan)+ two hotel mangers (Marriott and Hyatt) Five personal interviews with hotel guests

200 questionnaires distributed to loyal guests

To:

1) Assess the relevance of its conceptualization of marketing research operation;

2) Appraise the suitability of the terminology to the hotel context;

3) Make further suggestions, criticism and comments on the questionnaire and its facets; and

4) Validate the questionnaire Same as procedure one

To:

1) Ask guests give their comments and identify any problems in regard to the questionnaire; and

2) Interviews results used in pre-testing

To:

1) Modify and refine the questionnaire prior to the final survey; and

2) Perform proper analysis

In the fourth and final procedure, letters of formal invitation were mailed to the hotel managers in order to invite them to distribute the amended questionnaires to their

guests (see Appendix A.2). Two sets of discussions were undertaken to determine the best way to approach the guests to fill out the questionnaires. Discussion with both academics (procedure one) and hotel managers (procedure two) identified that the most appropriate way was to distribute the questionnaires when guests checked in, and collect them when guests checked out. This would give guests the chance to take the survey with them and fill it out during their stay. In total, 200 questionnaires were distributed to guests by front desk staff at each of the above-mentioned hotels. This procedure was conducted during June and July 2005. Twenty-five usable questionnaires were received (13 % response rate). To assess the reliability of the measures, Cronbach’s coefficient alpha was examined, showing that all scale items had high alpha scores exceeding .70. Following the reliability assessment, the purpose was to assess convergent and discriminant validity of items by using confirmatory factor analysis. However, it was not possible to conduct this due to the small sample size (N=26). Hence, validity assessment was conducted after the final data collection, and discussed as part of SEM in the next chapter.

Further to the above empirical results, respondents’ answers identified that there was a need for additional modifications. For instance, a question related to the job title was deleted as most respondents did not answer it and the income categories were changed to be more suited to the Arabic context. In addition, the pre-test confirmed that giving questionnaires through front desk staff at targeted hotels is the most appropriate way to approach potential respondents.

In total of all above four pre-test procedures, minor changes to statement wording and layout were made to the instrument to ensure that the questions were readily understood by all respondents (Zikmund, 2003). As no major modifications were made to the instrument, a further pre-test was considered unnecessary. A copy of the final survey instrument used for this thesis is provided in Appendix B.