Control de Calidad Interno

The questionnaire was created in order to measure the preference, as defined by intent and desire, of installing energy efficiency technologies in the home and/or adopting energy efficiency behaviours. In total, 18 energy efficiency technologies and 7 behaviours were included in the questionnaire. The technologies were se-lected based on their inclusion in the UK’s Standard Assessment Procedure (SAP), that is the methodology used by the UK government for assessing and comparing the energy and environmental performance of dwellings [Department of Energy and Climate Change, 2012c]. The behaviours were inductively identified from the 198 general population interviews conducted in Manchester and Cardiff[Pelenur

& Cruickshank, 2011b]. Table 3.3 lists the specific technologies and behaviours included in the questionnaire.

For each technology and behaviour, the questionnaire used a 7 point Likert Item (from “strongly disagree” to “strongly agree”) to measure the response of the following questions: “I want to fit/adopt this measure/behaviour in my home”;

and “I intend to fit/adopt this measure/behaviour in the next 12 months.” Par-ticipants were also asked to tick a box if they had already installed/adopted the technology/behaviour in their home and to comment on any differences between their ‘intent’ and ‘desire’ responses, such as barriers, or motivations. If there were any technologies or behaviours unfamiliar to the participant, they were asked to

Table 3.3: Technologies and behaviours included in the questionnaire

Ground source heat pump Domestic Combined Heat and Power (CHP) Air source heat pump Energy ecient lighting

Passive lighting Micro-wind

Solar PV Solar thermal

Improved heating controls Radiator thermometers Behaviours

Seek energy saving advice (from energy companies or government)

Coordinate the time-of-use of appliances in order to minimise peak demand Turn appliances o completely rather than leave on stand-by

Get rid of unnecessary gadgets or appliances Consciously use less

Use lower temperature for washing machine Put on a jumper before turning up the heating

skip the question. Finally, the questionnaire asked a series of personal and house-hold demographic questions. At the end, each response was reviewed together by the researcher and the participant to ensure that they understood the differ-ence between ‘intent’ and ‘desire’ for installing/adopting energy efficient technolo-gies/behaviours. Similar to the Q-set, the questionnaire was also piloted in Cam-bridge.

Linking the questionnaire with the Q-study

In order to link the results of the questionnaire with the Q sorts, tests of associa-tions were carried out between the numerical Q factor loadings and the question-naire responses. For this purpose, other studies have used ANOVA, MANOVA, Pear-son’s correlation, and Path Analysis[Kubier, 2010; Thomas & Baas, 1996; Thomas et al., 1982, 1993]. However, since the questionnaire variables consisted of mul-tiple data types, a range of correlation measures and test statistics were used to investigate the relationship between the numerical Q factor loadings and the ques-tionnaire responses. For example, the technology intent and desire Likert Items were interpreted as ordinal while the type of home (a demographic variable) was

categorical (nominal). While there is considerable debate around the interpreta-tion of Likert Scales (which are composed from Likert Items), it is generally rec-ommended that individual Likert Items should be analysed as ordinal data[Carifio

& Perla, 2007; Jamieson, 2004]. Table 3.4 summarises the data types associated with each of the questionnaire variables, and the test of association or test statis-tic used to correlate each variable with the numerical (interval/scale continuous) factor loadings from the Q study.

Table 3.4: Tests of associations with interval continuous variable

Questionnaire variables Data type Test of association Technology/behaviour desire Ordinal - Likert Item Spearman's rho Technology/behaviour intent Ordinal - Likert Item Spearman's rho Installed/adopted (yes/no) Dichotomous nominal Point-biserial

Sex Dichotomous nominal Point-biserial

Age Ordinal Spearman's rho

Education level Nominal Anova (F Test)

Marital status Nominal Anova (F Test)

Household income Ordinal Spearman's rho

Tenure Nominal Anova (F Test)

Type of home Nominal Anova (F Test)

House age Ordinal Spearman's rho

Number of bedrooms Interval Pearson correlation

For all the listed tests of association, the null hypotheses H_o was no significant correlation between any of the questionnaire variables and the factor loadings, while the alternative hypothesis H₁ was the existence of any correlation between the variables. Since there were multiple hypothesis testing between variables, the resulting p-values from each test were adjusted to correct for multiple comparisons.

For this study, the Benjamini-Hochberg procedure (BH step-up) was used to adjust the final p-values and control the false discovery rate (FDR) , i.e. the expected pro-portion of incorrectly rejected null hypothesis (“false discovery”). FDR procedures are widely used in data rich fields such as: physics; weather mapping; and genet-ics, because the procedures have been shown to more powerful than comparable methods that control for the traditional familywise error rate (such as the Holm or Bonferroni method)[Abramovich & Benjamini, 1996; Weller et al., 1998; Yekutieli

& Benjamini, 1999]. The adjusted p-values help correct for errors introduced by multiple comparisons, and are a more accurate reflection of significance.

3.6.4 Seasonality

The Q-study and questionnaire were both conducted during summer; however, subjective viewpoints towards energy use may change in the winter, when the heating is on and outdoor temperatures have dropped. As such, a winter survey was carried out after the summer research with all 91 participants (46 in Manch-ester, and 45 in Cardiff). The aim of the survey was to investigate how the change in seasons affected viewpoints towards household energy use.

The survey was administered as a posted questionnaire with an introductory letter, and an offer to win £50 in Amazon vouchers for participating. In order to increase the returned response rate, the questionnaires were kept as simple as possible. In total, 5 questions were asked. All the questions except the fifth were repeated from the Q-study or the street interviews. Specifically, the first 4 questions were:

• In general, can you sum up your thoughts about energy use in your home (electricity/gas)? [Open ended answer]

• Is there anything you would like to change about how your household uses energy? If yes, what? Please include motivations for the change and/or barriers stopping you. [Open ended answer]

• How frequently do you think about your household energy use? [5 point Likert Item]

• Thinking about your home in the winter, how easy or difficult is it to keep your home warm when the heating is on? [4 point response scale]

The fifth question asked participants to read through the original 65 statements used in the Q-study and select the 5 statements they most agree with, and the 5 statements they most disagree with. Finally, a response box was provided for par-ticipants to explain the reasoning for their selections. This approach was selected since it was somewhat analogous to a Q-study but much more simplified; as op-posed to administering a second full Q-study by post, which may have been too onerous and unreliable for the participants.

In total, 91 winter surveys were posted and 34 were returned, 19 from Cardiff and 15 from Manchester, resulting in an overall response rate of 37% . This re-sponse return rate was within one standard deviation of a normal average for mailed out questionnaires, as identified by review papers employing response rate meta-analysis [Baruch, 1999; Baruch & Holtom, 2008]. From the 34 returned surveys, 3 were incorrectly filled out and could not be used.

The results from the winter survey were compared against the main summer study, and differences identified in order to enhance the discussion of the main study conclusions. However, since the winter survey was not a full Q Study, it was not possible, nor desirable, to verify the test-retest reliability with the sum-mer survey. Test-retest reliability asses the consistency of a measure from one time to another, i.e. administering the same test to the same sample on two different occasion [William Trochim, 2006]. For Q methodology, the Q-sort reliability co-efficients of a person with himself have been shown to normally range from 0.8 upward[Brown, 1980; Dennis, 1992; John Nicholas, 2011].