Table A11 Notes guiding quality management model adapted from (Walther, Sochaka & Kellam , 2013) and applied to this work.
Description Making the data Handling the data
How can we improve the research findings’ capacity to appropriately capture and represent aspects of the social reality observed?
Do we get to see what we think we see?
Is the researcher’s version grounded in the versions of the participants’? Do we call things by the right names?
Theoretical validation:
Do the concepts and relationships of the theory appropriately correspond to the social reality under investigation?
The research process needs to be able to capture the full extent of the social reality studied.
Interpretations need to reflect the coherence and complexity of the social reality under investigation.
• How interactions with high efficiency homes affect social energy literacy. Case study homes were selected by their design features and inhabitants willingness to participate. • The case studies include survey, observation, and interview data. Each data collection process follows a reviewed protocol which focuses on a specific set of energy literacy concepts. Lessons learned are entirely generated by the participant requiring them to genuinely express related experiences. • Interviews provide the depth and detail in this study and are assisted by observations and survey data that allow the researcher to search for a wide breadth of potential experiences. This study attempts to understand a large** phenomenon by having participants extend the researchers experience though their memories and thoughts.
• The entire analysis
rests on energy literacy concepts that relate to the research questions.
Observations of interactions with home design features and reported
learning/sharing/applying is investigated in more detail in interview questions which require participants to articulate their meanings of observations and answers. • Descriptive coding and then pattern coding will be used to allow flexibility to navigate the complex reality I am investigating.
• By tying my descriptive and pattern codes to “theories” I will be anchoring my study into multiple sought after contexts.
Procedural validation:
Which features of the research design improve the fit
Strategies need to be implemented in the research design to mitigate threats to contextual validation. Features, processes, mechanisms
Processes need to be implemented to mitigate risks of mis-
between reality and the theory generated? constructing the participants’ reality in the researcher’s interpretatio ns.
• The survey and observation data informing
the interview protocol is a procedural element to increase data quality; there is an explicit structural connection between the high efficiency home survey, in home field observation protocol, and interview protocol.
• Our process of transforming survey responses into an individualized interview guide improves fit between the theory and reality we investigated: we first asked an open-ended question supported by our guide of prompts from the survey responses. “Please provide me an example of how this home does a very good job teaching that energy can be calculated and monitored”
The interview protocol was designed to maximize quality via interview
preparation, including explaining the importance of the research to the participants.
• Using verbatim transcript
data, parallel coding, member checking
Sought advice/peer debriefing from *** external researchers with expertise in theoretical and methodological aspects of the project
Communicative validation:
Is the knowledge socially constructed within the relevant communication community?
The data gathering needs to capture the respondents’ inter-subjective
reality. The researcher’s
abstract interpretations need to be grounded in the accounts of the participants. The knowledge produced needs to be represented in accordance with the meaning conventions of the research community.
• Used each participant’s observation and survey data to form individual semi- structured interview guides.
Framed questions to avoid energy jargon to ensure that participants
• Listening to voice
recordings of the participants ensured that we could “hear” the participants voices as we
authentically constructed their own meanings.
Repeatedly verified if we were
obtaining authentic dialogue to ensure that the reality of experience of all our interviewees came forward
• Examined the possible occurrence of our
own personal bias during the interview leading us to predetermine conclusions and preventing the co-construction of actual meaning here.
• Ensure a smooth transition of data of the participant experience to that of the research community
handled the data; reassured us that although we were no longer talking to them, we adhered close to their meaning and close to the data as we shifted audiences to the research community for
transparency
• Co-constructing meaning
making within the research group, two researchers doing interviews together links the research team; research team is carrying meaning from participants’ voices to research
community audience.
• Used language consistent
with the customs of the research or research community (used Department of Energy energy literacy language in the analysis and writing the paper), in order to co- construct meaning with the research community.
• Continuously revisited verbatim transcripts and used illustrative quotes from participants .
• Carefully established
findings with our original research question and theoretical considerations of learning, sharing, and applying energy literacy concepts.
• Presented raw transcript data to outside experts to validate differences in “the telling” and “the told” (peer debriefing for appropriateness of the methods and conclusions).
Pragmatic validation:
Do the concepts and knowledge claims withstand exposure
The concepts underlying the research design need to be
compatible with reality in the field.
The knowledge produced needs to be meaningful in the social context under investigation.
to the reality investigated?
• The mixed methods approach enhanced
additional understanding that could not be achieved with one method alone.
• We examined the theoretical constructs
brought to the study to the theoretical constructs generated from the study, to determine asking if they “survive” exposure to empirical reality. That is, do they have fidelity in generating response or promoting understanding? Do they resonate or have meaning with reality?
o The assumption that people are learn from exposure was kept in check by requiring participants to come up with their own examples of learning.
o The assumptions that people will share or apply their knowledge
• The assumption that knowledge is
developed due to interaction with a high efficiency home is addressed by phrasing the questions with respect to the home and requiring examples from the home.
• The pass system used in
analyzing the data demonstrates that our interpretation was grounded in what
participants said and in the theory to which the research community has to compare.
• From here, future studies
may be directed at the investigation of the disconnect with the theory and the social reality experienced at high efficiency homes
• The true pragmatic
validation will come with publication of the findings and test of their
transferability.
Process reliability:
How can the research process be made as
independent as possible from random influences?
The data needs to be collected and recorded in a dependable way
Procedures for generating and representing knowledge need to be established and documented. • Transcripts were checked and corrected
while listening to digital recordings
• Random influences were minimized through protocols such as the way the interviewers deliberately presented this as University sponsored work, and communication about participants’ importance to the overall project.
• Interviews were conducted in the case study
homes and in libraries. The audio was recorded and transcribed allowing the researcher to take notes during the interview of body gestures, and other situational occurrences that the audio may not be able to fully capture.
• Interview data and
interpretation were
compared to survey answers and observations before, during analysis
Parallel coding, verbatim analysis
• Memo’s were created and discussed with my research collaborators throughout the process.
References
ABET. (n.d.). Criteria for Accrediting Engineering Programs, 2016 – 2017 | ABET. Retrieved from http://www.abet.org/accreditation/accreditation-criteria/criteria-for-accrediting- engineering-programs-2016-2017/
Allcott, Hunt, and Sendhil Mullainathan. “Behavior and Energy Policy.” Science 327, no. 5970 (March 5, 2010): 1204–5. doi:10.1126/science.1180775.
Allen, S. (2004). Designs for learning: Studying science museum exhibits that do more than entertain. Science Education, 88(S1), S17–S33. http://doi.org/10.1002/sce.20016 Anderson, D., & Lucas, K. B. (1997). The effectiveness of orienting students to the physical
features of a science museum prior to visitation. Research in Science Education, 27(4), 485–495. http://doi.org/10.1007/BF02461476
Anderson, J. R., Reder, L. M., & Simon, H. A. (1996). Situated Learning and Education.
Educational Researcher, 25(4), 5–11. http://doi.org/10.3102/0013189X025004005
Andres, R., & Loudermilk, M. (2011). Living Green Improving Military Energy Behavior & Culture : Center for a Better Life. Retrieved September 11, 2013, from
http://www.centerforabetterlife.com/eng/magazine/article_detail.lasso?id=210 ASCE. (n.d.). Code of Ethics | ASCE. Retrieved July 15, 2016, from
http://www.asce.org/code-of-ethics/
Attari, S. Z., DeKay, M. L., Davidson, C. I., & Bruin, W. B. de. (2010). Public perceptions of energy consumption and savings. Proceedings of the National Academy of Sciences,
Barr, S., Gilg, A. W., & Ford, N. (2005). The household energy gap: examining the divide between habitual- and purchase-related conservation behaviours. Energy Policy, 33(11), 1425–1444. http://doi.org/10.1016/j.enpol.2003.12.016
Becker, H. S. (1977). Sociological Work: Method and Substance. Transaction Books.
Cai, J. (n.d.). Parental Roles in Students’ Learning of Mathematics. Retrieved April 28, 2016, from http://files.eric.ed.gov/fulltext/ED412087.pdf
DeWaters, J., & Powers, S. (2013). Establishing Measurement Criteria for an Energy Literacy Questionnaire. The Journal of Environmental Education, 44(1), 38–55.
http://doi.org/10.1080/00958964.2012.711378
DOE. (2012, July). Energy Literacy: Essential Principles and Fundamental Concepts for Energy Education. U.S. Department of Energy. Retrieved from
www1.eere.energy.gov/education/energy_literacy.html.
DOE. (n.d.). Whole-House Systems Approach | Department of Energy. Retrieved July 12, 2016, from http://energy.gov/energysaver/whole-house-systems-approach
Ehrhardt-Martinez, K., Donnelly, K. A., & Laitner, J. A. (2010). Advanced Metering Initiatives and Residential Feedback Programs: A Meta-Review for Household Electricity-Saving Opportunities. 2010. In American Council for an Energy-Efficient
Economy (p. 140).
EIA. (n.d.-a). Annual Energy Review. Retrieved April 7, 2016, from http://www.eia.gov/totalenergy/data/annual/pdf/sec1_13.pdf
EIA. (n.d.-b). Energy Use in Homes - Energy Explained, Your Guide To Understanding Energy - Energy Information Administration. Retrieved July 7, 2016, from
http://www.eia.gov/EnergyExplained/?page=us_energy_homes#tab2
Gaither, M. (2009). Homeschooling in the USA Past, present and future. Theory and Research
in Education, 7(3), 331–346. http://doi.org/10.1177/1477878509343741
Gieryn, T. F. (2002). What Buildings Do. Theory and Society, 31(1), 35–74. http://doi.org/10.2307/658136
Gould, K. A., & Lewis, T. L. (2009). Twenty lessons in environmental sociology. Oxford University Press.
Graziano, M., & Gillingham, K. (2015). Spatial patterns of solar photovoltaic system adoption: The influence of neighbors and the built environment. Journal of Economic
Geography, 15(4), 815–839. http://doi.org/10.1093/jeg/lbu036
GREDLER, M. E. (2009). Hiding in Plain Sight: The Stages of Mastery/Self-Regulation in Vygotsky’s Cultural-Historical Theory. Educational Psychologist, 44(1), 1–19. http://doi.org/10.1080/00461520802616259
GRMI. (n.d.). PowerHouse Experience. Retrieved July 15, 2016, from http://www.gmri.org/our-work/science-education/powerhouse
Heath, C., & Heath, D. (2008). Made to stick: why some ideas survive and others die. New York: Random House.
Hyslop-Margison, E. J., & Strobel, J. (2007). Constructivism and Education:
Misunderstandings and Pedagogical Implications. The Teacher Educator, 43(1), 72–86. http://doi.org/10.1080/08878730701728945
(ISWG) The Interagency Sustainability Working Group. “HIGH PERFORMANCE and SUSTAINABLE BUILDINGS GUIDANCE,” 2008.
http://www1.eere.energy.gov/femp/pdfs/guidance_hpsb.pdf.
Janda, K. B. (2011). Buildings don’t use energy: people do. Architectural Science Review,
54(1), 15–22. http://doi.org/10.3763/asre.2009.0050
Laitner, J. A., & Ehrhardt-Martinez, K. (2009). Examining the scale of the behaviour energy efficiency continuum. Retrieved from
https://www.etde.org/etdeweb/details_open.jsp?osti_id=967609
Lin, N. (2002). Social Capital: A Theory of Social Structure and Action (Structural Analysis in
the Social Sciences). Cambridge University Press. Retrieved from
http://www.amazon.ca/exec/obidos/redirect?tag=citeulike09- 20&path=ASIN/052152167X
Maxwell, J. A. (2012). Qualitative Research Design: An Interactive Approach: An Interactive
Approach. SAGE.
McMakin, A. H., Malone, E. L., & Lundgren, R. E. (2002). Motivating Residents to Conserve Energy without Financial Incentives. Environment and Behavior, 34(6), 848–863.
http://doi.org/10.1177/001391602237252
Miles, M. B., Huberman, A. M., & Saldaña, J. (2013). Qualitative Data Analysis: A Methods
Sourcebook. SAGE.
Tom Murse. “The History of Solar Panels on The White House.” About.com News & Issues, December 4, 2015. http://usgovinfo.about.com/od/thepresidentandcabinet/tp/History-of- White-House-Solar-Panels.htm.
National Research Council. (2000). How People Learn: Brain, Mind, Experience, and School:
Expanded Edition. National Academies Press. Retrieved from
http://www.nap.edu/catalog/9853/how-people-learn-brain-mind-experience-and-school- expanded-edition
Next Generation Science Standards: For States, By States. (2013). Washington, D.C.:
National Academies Press. Retrieved from http://www.nap.edu/catalog/18290 Orr, D. W. (1997). Architecture as Pedagogy II. Conservation Biology, 11(3), 597–600.
http://doi.org/10.1046/j.1523-1739.1997.011003597.x
Puntambekar, S., & Hubscher, R. (2005). Tools for Scaffolding Students in a Complex
Learning Environment: What Have We Gained and What Have We Missed? Educational
Psychologist, 40(1), 1–12. http://doi.org/10.1207/s15326985ep4001_1
Riley, D. (2011). Engineering Thermodynamics and 21st Century Energy Problems: A
Textbook Companion for Student Engagement. Morgan & Claypool Publishers.
Schunk, D. H. (2012). Learning Theories: An Educational Perspective. Pearson.
Sovacool, B. K. (2009). The cultural barriers to renewable energy and energy efficiency in the United States. Technology in Society, 31(4), 365–373.
http://doi.org/10.1016/j.techsoc.2009.10.009
The Council on Environmental Quality. “Guiding Principles for Sustainable Federal Buildings and Associated Instructions,” February 2016.
https://www.whitehouse.gov/sites/default/files/docs/guiding_principles_for_sustainable_f ederal_buildings_and_associated_instructions_february_2016.pdf.
Torcellini, P., Pless, S., Deru, M., & Crawley, D. (2006). Zero energy buildings: A critical look at the definition.
Toth, N., Little, L., Read, J. C., Fitton, D., & Horton, M. (2013). Understanding teen attitudes towards energy consumption. Journal of Environmental Psychology, 34, 36–44.
http://doi.org/10.1016/j.jenvp.2012.12.001
Tudge, J. R. H., & Scrimsher, S. (2003). Lev S. Vygotsky on education: A cultural-historical, interpersonal, and individual approach to development. Educational Psychology: A
Century of Contributions, 207–228.
Walther, J., & Sochacka, N. (2014). Qualifying qualitative research quality (The Q3 project): An interactive discourse around research quality in interpretive approaches to engineering education research. In 2014 IEEE Frontiers in Education Conference (FIE) Proceedings (pp. 1–4). http://doi.org/10.1109/FIE.2014.7043988
Washington, N. G. S. P. O. B. 98199, 38.90531943278526, D. 20090-8199 U., & 800-647- 5463, -77 0376992225647. (2013). Urban Ski Slope to Raise Profile of Europe’s Waste- to-Energy Drive. Retrieved January 12, 2015, from
http://news.nationalgeographic.com/news/energy/2013/08/130801-amager-bakke-europe- waste-to-energy/
Webb, D., Soutar, G. N., Mazzarol, T., & Saldaris, P. (2013). Self-determination theory and consumer behavioural change: Evidence from a household energy-saving behaviour study. Journal of Environmental Psychology, 35, 59–66.
World Energy Council. “2016 World Energy Issues Monitor.”
https://www.worldenergy.org/publications/2016/world-energy-issues-monitor-2016/. Yates, S. M., & Aronson, E. (1983). A social psychological perspective on energy
conservation in residential buildings. American Psychologist, 38(4), 435–444. http://doi.org/10.1037/0003-066X.38.4.435
Zografakis, N., Menegaki, A. N., & Tsagarakis, K. P. (2008). Effective education for energy efficiency. Energy Policy, 36(8), 3226–3232. http://doi.org/10.1016/j.enpol.2008.04.021