Arquitectura de hardware

Questionnaire items were, for the majority part, adopted or adapted from prior related research (see Chapter 3, Table 3.3), however some measures were self-developed. Two general types of questions are used in questionnaires – closed-ended and open-ended [275]; whereby the former require a selection of one or several of given response options, while the latter enable participants to express their views in their own words. The questionnaires in this research mainly comprised closed-ended questions, which included: dichotomous, multiple choice, multiple response, and grid or rating scale. The questionnaires were divided into several sections, each addressing a single research construct. Questionnaire instruments used in this research are shown in Appendix B. The first section generally collected user background data (apart from Study 1 where it was collected at the end). This included a mixture of demographic questions (where users were provided a textbox to type in their response), e.g., age, and multiple-choice questions, e.g., how cancer affected the respondent.

It was normally the second and the last section that presented the emotion measuring instrument for self-reporting emotions experienced before using the PORT website and after interacting with the website, respectively. Section 4.3.1.2 of this chapter explains the instrument in detail.

The third section presented the instrument for evaluating personalisation features, in terms of usefulness. This was a grid-type question, which listed the personalisation features in rows and the rating scale in columns. Usefulness was rated on a 5-point scale (1 to 5) for level of agreement in Study 3 and 4, and on a 5-point scale (-2 to 2) for level of usefulness in Study 2 and 5. The aim was to reduce the effect of the scale used on participant responses, as well as to simplify and clarify response options, particularly in the follow-up study (Study 5). In Study 1, availability and preference for a feature was expressed by selecting from multiple choices.

The features that were evaluated in Study 1 were the adaptive and personalisation features commonly identified in the adaptive hypermedia literature [75, 79, 80] and

84

studies on online personalisation [77]. In Study 2 to Study 5 the personalisation features that were evaluated were the ones that were implemented on the PORT website. Therefore in these studies, participants were inquired on their perception of the personalisation features they experienced on the PORT website. The lengthy list of personalisation features (in all studies more than 20 features, i.e., questionnaire items), was randomised in the last two studies (Study 4 and 5), to account for the factors of fatigue and participants’ inclination for straight-lining. The complete list of personalisation features evaluated in the five studies is provided in Appendix C.

Commonly, the third section also included an instrument for measuring a preference for an approach to personalisation. In Study 1 users were asked which approach to providing personalisation features (system controlled or user controlled) they preferred for each feature evaluated. In Study 3, participants expressed their perceived preference

towards adaptivity or adaptability, via five questionnaire items stating either an adaptive or adaptable personalisation service.

Note that, in measuring the factor preference for adaptivity vs. adaptability, reverse coding was applied. As can be seen in Appendix B, three of the items of this factor expressed a preference for adaptability, and two a positive attitude toward adaptivity. The factor itself was measured as an overall attitude toward adaptability. Thereby, to ensure that all the five items express a positive attitude towards adaptability, the two adaptivity oriented items were reverse coded. For example, the item: I prefer the system to tailor its features automatically, with a response Yes, was re-coded into: I do not prefer the system

to tailor its features automatically, with a response No.

Finally, in the last two studies, usefulness of adaptivity and usefulness of adaptability

were measured. The latter two constructs were measured by classifying the list of items used for the factor usefulness of personalisation features into website services that perform automatic tailoring (adaptivity) and, on the other hand, those that require user customisation (adaptability). Chapter 7 (Section 7.1.2) provides a detailed list of the items of the two factors. Furthermore, the same 5-point rating scale for the level of agreement was used.

85

The fourth section presented the instrument for measuring satisfaction with website personalisation. This construct’s items were adapted from the following factors that were explored in previous studies: personalisation quality [221], perceived personalisation

[126], satisfaction and usefulness [222], site quality [223], personalisation [83], and

motivation [219]. Nevertheless, the instrument used here was adjusted to reflect the type of personalisation offered on the PORT website. Hence, users evaluated website appearance, ease to find and relevance of content, matching the content to user interests and emotions, etc. (see Appendix Bfor detailed items and scales). A 5-point Likert scale for the level of agreement was used in this case.

The instrument for measuring website reuse intentions was presented in a separate questionnaire section. The items for this construct were adopted from several studies that measured: intention to purchase [83, 125], behavioural intentions [90], likelihood of usage [221], and the intention to adopt [126]. In Study 2 and 3, the reuse intentions instrument comprised five items, these were reduced to three in Study 4. The two excluded items (‘I intend to revisit’, and ‘I intend to use frequently’) were comprised in the remaining items (‘I intend to use again’). The reduction of items did not significantly affect the factor’s reliability, which was sufficiently high with three items, however it simplified and shortened the survey and reduced the demand on participants. A 5-point Likert scale for the level of agreement was used here as well.

Website usability instrument was included in the questionnaires for Study 2 to Study 5. In all the studies, a 5-point Likert scale was used to rate the level of agreement with the items measured. The instruments used in Study 2 and 3 were based on a subset of questions selected from the Computer Usability Questionnaire (CSUQ) [238] and User Interface Satisfaction (QUIS) [239] (see Appendix B). These questionnaires contain a larger set of questions (19 and 27 respectively), however, it is not an uncommon practice for a subset of questions to be selected [82]. The last two studies used the System Usability Scale (SUS) [220] instrument. SUS has fewer items (10 in total), enables calculating a score and easily comparing usability across different website versions or other similar systems. Moreover, using different usability instruments enabled comparing the consistency of the findings across the studies (Chapter 6).

86

The previously mentioned reverse coding/scoring was also applied to the usability items (see Appendix B). In calculating the usability score using the SUS instrument, reverse coding is incorporated in the calculation. However, in Study 2 and Study 3, I had to reverse code the negatively worded items. For example, the item: I did not feel confident using the PORT website, was reworded into: I felt confident using the website. Thereby, the scale was reverse scored, so that the previous strongly agree response, was a strongly disagree response on the re-coded item, and vice versa. Thus, the values of the scale 1-5, all referred to the same type of response for all usability items, and all items were worded in the same direction.

At the end of a survey, an open-ended question was presented. Participants were invited to provide feedback, comments, remarks, complaints or suggestions they have with regard to their experience in using the PORT website, or any general suggestions.

The longer surveys, as the one in Study 3, additionally included trap questions in different sections of the questionnaire. These were for example – What is the first letter of the alphabet? – which required obvious answers. They served for validation purposes, to assist in filtering out random responses.