The following paragraph discusses the pilot study subjects, findings, and resulting improvements.
3.7.1 Pilot Study Subjects
The pilot studies took place from January to April, 2007. The subjects for the pilot studies were defined into three different and progressive studies. These three studies were (1) two master students at ESHotel in Paris, France; (2) five students at Ecole hoteliere de Lausanne, Switzerland; and (3) ten students from Pepperdine University, USA.
3.7.2 Process modifications based on pilot studies
Although the first pilot study subjects are French hotel school students, they had limited internet experiences, and no online shopping experiences. As these two subjects had never been to Sidestep.com, the researcher asked them to find a hotel in London to go over the functions of Sidestep.com. During the process, the researcher first gave the subjects time to scroll around the web site, showing the information available, such as map and customer
reviews for individual hotels, and asked subjects to try both the Smartsort function and the comparison function. This trial task was added for the 2nd and 3rd group pilot study subjects, and will be added to the official subjects. This pre-pilot study task also checked the functionality of a web cam to record the process.
When conducting the selection task, these two subjects just scanned through the first page of the first output, read the brand names and prices, and made their decisions. Uneasiness, lack of knowledge and experience in hotel features were observed when these two subjects did the selection task. For example, one subject used the comparison function, and decided to select a hotel because the hotel’s description showed ‘TV’. However this was not observed with other subjects. It was speculated that the lack of both internet and online purchase experiences were the reasons for their behaviours. The interview and the conjoint analysis process went well.
During the first pilot study, a web cam was used to tape the process. As the web cam was between the subject and the laptop, it prevented the subjects from moving the laptop to their preferred positions. Furthermore, the web cam video had light reflection. During the process, the original observation form was used but it proved in effective, as subjects moved fast from one screen to another. It proved impossible to keep track of every click during the pilot study process.
After the first pilot study, the recording process was reconsidered. Pan & Fesenmaier (2003) reported using an online camcorder (TechSmith®, C a m ta s ia ^ M ) and iO p u s ® S T A R R T M to record subject’s keystrokes. It was found the cost of TechSmith® USD 298.95 to be too high.
However iO p u s ® S T A R R T M was downloaded for a free trial and tested. In theory, since
iO p u s@ S T A R R .T M records and repeats every stroke during the process, it is possible to keep track of the mouse movements, hotels and attributes the subjects clicked. However, due to the live nature of Sidestep.com, it was found that both the sidestep.com database and internet speed differed. Therefore, iO p u s @ S T A R R T M was not suitable for the recording process.
Desk Screen Record 5 was then investigated. Desk Screen Record can record the screen during the process, turn it into an AVI file, which can be played by Windows Media Player. It costs only USD 29.95. This was tested and the AVI file reviewed. The output was satisfactory and the file also records the time. It also overcame the problems - light reflection and movements - associated with a web cam. In addition, since the online hotel selection task is recorded the data on Table 3.2 (page 136) can be recorded after the entire data collection process.
The second group of pilot studies was conducted with students from Ecole hoteliere de Lausanne. Since Desk Screen Record offered 10 free trials, the subjects were asked to download the program and used their laptops to conduct the pilot study. After the trial task, the subject started the accommodation selection process. Both the trial task and the hotel selection process were taped under one file. After the hotel selection task, the AVI file was saved to a movable hard drive under the subject’s name.
The first subject in the second group had two situations. First, the subject wanted to specify the areas (location) of the properties. Based on the previous study conducted by the researcher, the subject was reminded that the difference between “off the Strip” and “the Strip” (Las Vegas) lies in whether the hotel’s main entrance in “on the Strip”. Second, after the first subject clicked on both “off the Strip” and “the Strip”, she received identification of
31 hotels, which were all motels. It was decided this was due to the dates used, so this was changed to Oct 15 - 19, 2007. Thus, the subject conducted another search with the new travel date. A similar situation occurred with the second subject in this pilot study group.
The numbers of available properties during two travel periods were 78 and 77. This too was considered to be low. It was decided to change the travel date to Dec 3 to 7 in 2007 for the remaining pilot study subjects.
After these two subjects, it was also decided to standardize the process by adding a trail task to the selection task in order to familiarize the subject with the website. A standardized instruction was written and given to all other subjects. The trial task asked the subject to find a hotel for 2 persons between Dec 3 and 4, 2007, in London. No specific budget was given.
The official task was to find a Las Vegas hotel for 2 persons between Dec 3 and 7, 2007. The budget was USD 1,000. The other subjects in the second group all had extensive internet experiences and online purchasing experiences. The processes went well.
The third group of pilot studies took place in the computer lab of the Pepperdine University dormitory in Lausanne. A total 10 students signed up for the pilot study. As they are students from an American university, three of them had been to Las Vegas already. When asked if they had preferred hotel to stay for their next Las Vegas trip, they did not. The advantages and disadvantages of using Pepperdine University students as official subjects were as follows. The advantages were that they speak English, and have experiences arranging and purchasing travel products online. The disadvantage was the similarity among subjects. In fact, all 10 subjects were sophomore students with different majors. They had either 1 or 2 study abroad semesters, and travelled extensively during their study abroad
semesters. They all started using the internet at about 7-th grade, had online purchase experience, and used the internet to plan and purchase their hotel stays.
Several subjects in the third pilot study used the comparison function. However, the researcher found that when the subject clicked on more than 4 properties, the subject had difficulties viewing the information. The web site presented the competing properties by property name (column title) and attributes (row titles). However, if there were more than 4 properties, the screen could not show all properties. If the subject scrolled the screen to the right in order to read the information, the attributes would be off the screen. Therefore, the researcher reminded subjects to click on two or three properties to compare each time.
Although the subjects could keep and change properties to compare, subjects tended not to do it. In other words, this advice may have influenced the subjects to cut down their choice set.
To prevent the interference, it was decided not to remind the subjects in the in the main study.
The third pilot also trialled working with two subjects at a time. Under this situation, the first subject followed the normal procedure; while the second subject to started with the conjoint analysis task, followed by the accommodation selection task and the interview. Although this reduced the time taken overall, it was complex managing two subjects at a time. So it was decide not to do this for the main study.
The researcher used the observation form and interview form to record the data from pilot studies and made modifications. The original observation form was developed based on the assumption that subjects will use Smartsort function first, and comparison function later.
However, most subjects didn’t follow this sequence, but switch between Smartsort and comparison functions. Therefore, the original observation form can not capture the process
dynamics. The author decided to replace the observation form with detailed written descriptions. In addition, these subjects tend to talk during the selection process to explain to the researcher what they are doing or what they are thinking. The author noted down these conversations, which are useful to refresh the author’s memory. The author decided to redesign the observation form into a document with certain headings to keep better records of the observations and conversations as shown in Table 3.2 (page 136).
The similarity of the third pilot study members resulted in changing the interview process to a questionnaire survey. These subjects were relatively homogeneous, ie sophomore students about age 20 years old. Since these subjects also had almost identical experiences, it was easier to use a questionnaire to collect data.
The hotel selection task is influenced by the website environment. Chen and Jones (2006) conducted a detailed evaluation of website functionality. Sidestep.com was originally identified as the most suitable for conducting the hotel selection tasks for the three pilot study groups. Unfortunately, after the third pilot study, Sidestep.com, (along with Hotels.com, and Expedia) removed the comparison function. Referring to Figure 3.6, Travelocity.com offered similar attributes as Sidestep.com, and had the comparison function, and so it was adopted to replace Sidestep.com.
Figure 3.6 Attributes Screenshots ofSideStep.com and Travelocity.com desktop computers at Pepperdine University for the accommodation selection task.
(2) Desktop Screen Record program replaced the use o f a web cam to record the selection task. The record was saved as an AVI file, and could be viewed by Windows Media Player.
(3) Study subjects were Pepperdine University students.
(4) The observation form was modified from a check-list form to a transcript format.
(5) The interview process was replaced by a questionnaire to facilitate recording and analysis.
(6) The website for the hotel selection task was conducted by using Travelocity.com, instead o f Sidestep.com.