This study consisted of two phases: Phase I, during which the survey was administered; and Phase II, during which the selected group of participants were interviewed. This combination of survey and interview data provided a comprehensive view of faculty perceptions of online learning in engineering education from multiple perspectives.
SURVEY
Phase I of this study consisted of a survey of engineering faculty. The online survey instrument was developed to collect data to address the primary research question.
The survey items were developed based on prior research and theories related to technology adoption, engineering education, online courses in general, and online engineering courses in particular as described in Chapter 2.
The survey consisted of five major sections. The first section consisted of the introduction to the survey and the collection of individual and institutional demographic information. The second section explored respondents’ general perceptions about learning and teaching in an online environment and of whether certain topics can be effectively delivered in online courses. Sections three and four explored the concepts of perceived importance and effectiveness from the UTAUT and TAM models of technology adoption as applied to the important aspects of engineering education identified in the review of the literature. Section three asked respondents about their perceptions of the importance of various educational aspects and pedagogical methods used in engineering courses and section four gathered perceptions of effectiveness of the same factors as section three in relation to their use in engineering courses delivered online. Finally, section five explored perceptions of faculty support and barriers to implementation of online engineering courses. The full survey instrument is included as Appendix A.
The first section consisted of the introduction to the survey providing information about the methods and IRB requirements and nine questions designed to collect individual and institutional demographic information, such as age, gender, undergraduate or graduate focus, engineering discipline (civil, mechanical, electrical, etc.), type of institution (public or private), size of engineering program in terms of total enrollment, delivery method of engineering labs (face-to-face, remote, virtual or simulations), and experience with online courses as indicated by number of engineering courses taught by delivery method (online, blended, face-to-face, and simultaneous). These various
demographic factors have been shown to be relevant in various models of technology adoption such as the UTAUT (Venkatesh et al., 2003) as well as in other research into adoption and implementation of online classes (Allen & Seaman, 2008, 2010, 2011, 2013, 2014; Osborne et al., 2009).
Sections two through five of the survey contained the non-demographic perception questions. In general, these questions were delivered as statements with responses provided via a 5-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). The 5-point scale was selected to follow the format used in other research upon which this work is based. While respondents had the option to not answer an item, there was no option for ‘not applicable.’ Each survey item was designed to be independent and considered to represent a single factor. Due to instrument length and the number of different factors being researched, there were no intentionally redundant items for each concept. This is based on the use of individual items in similar research (Allen & Seaman, 2008, 2010, 2011, 2013, 2014; Osborne, 2010; Osborne et al., 2009).
The second section contained two main questions, each with multiple sub-items. The first question was based on a survey instrument developed by Osborne et al. (2009) to investigate general faculty perceptions of online courses. Since the research by Osborne et al. did not focus specifically on engineering courses, but rather included participants from social science programs, appropriate modifications were made as needed to adapt to this research topic. Many of these questions or variations were also used in the pilot study. There were 16 individual items presented in a list, with each phrased as a statement and responses measured using a 5-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). An example of this question and its sub-items is as follows:
What are your perceptions of online courses in general?
- Online courses in engineering are easier for students than face-to-face courses.
- Online courses in non-engineering topics are easier for students than face-to- face courses.
The second question contained six sub-items phrased as statements to gather perceptions of which general types of topics can be effectively taught online, such as technical topics, engineering design, or courses heavy in mathematics. Responses were on a 5-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). An example of this question and its sub-items is as follows:
Can these topics be effectively delivered in online courses? - Engineering theory courses can be effectively taught online. - Engineering design courses can be effectively taught online. - Engineering labs can be effectively taught online.
The third section consisted of a single question intended to explore the importance of certain factors in engineering education in general, and not only in an online context. These factors were developed based on information from the literature concerning important aspects and pedagogical approaches used in engineering education, such as design projects, labs, and real-world problems, as presented and discussed in Chapter 2. The question listed the 16 factors and asked the respondent if they are considered to be important in engineering courses. All responses were provided on a 5-point Likert scale
(1=Strongly Disagree, 5 = Strongly Agree). An example of this question and its sub- items is as follows:
This item is important in teaching ENGINEERING courses. - Project-based learning activities
- Lab activities - Team activities
The fourth section was similar to the third section, consisting of a single question intended to explore the perceived effectiveness of the same 16 factors included in section three as they relate to the online delivery of engineering courses. Perceived effectiveness is a significant factor in the adoption of technologies as per the UTAUT model (Venkatesh et al., 2003). All responses were provided on a 5-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). The items related to effectiveness, when combined with the items in section three relating to importance, allowed for comparisons and identification of factors that might be barriers to implementation of online courses in engineering. An example of this question and its sub-items is as follows:
This item can be effectively delivered or performed in an online format for ENGINEERING courses:
- Project-based learning activities - Lab activities
The fifth section explored various issues related to faculty support and potential barriers to implementation of online courses as described in the literature in Chapter 2 and incorporated some specific items from other survey instruments (Allen & Seaman, 2008, 2010, 2011, 2013, 2014). This section included 11 sub-items phrased as statements and rated on a 5-point Likert scale (1=Strongly Disagree, 5 = Strongly Agree). An example of this question and its sub-items is as follows:
What are your perceptions of support or barriers in teaching online courses? - It takes more time and effort to teach an online engineering course than a
face-to-face course.
- Faculty have appropriate technical support to develop online engineering courses.
INTERVIEWS
Phase II of this research consisted of semi-structured interviews based on significant and interesting findings from Phase I. After the collection and evaluation of the data from Phase I, interview questions for Phase II were developed to further explore and expand upon interesting or challenging findings from Phase I. These interviews helped explain and expand the understanding of the statistical results using the rich data provided by the individual real-world experiences and opinions of engineering faculty members. A similar process was implemented and found to be effective in the pilot study.
The interview consisted of eleven items: three regarding introductions and demographics, seven items exploring significant results from the quantitative survey, and an open-ended question at the end of the interview to allow respondents the option to
share any additional thoughts about online engineering education. Some of the interview items consisted of sub-questions on a particular topic as needed to fully explore the survey results. The full interview instrument is included as Appendix B. Examples of questions from the interview instrument are as follows:
- The final section of the survey asked about specific barriers to implementation of online engineering courses or programs. What do you think about the following results and why?
a. Respondents felt that faculty are not being compensated for developing and teaching online engineering courses. b. Respondents felt that faculty are not receiving appropriate
training in the development of online engineering courses. c. Two questions correlate and indicate that a lack of
acceptance by faculty is a barrier and that faculty do not accept the value and legitimacy of online engineering education.
Once the interview questions were developed, all interviewees received the same items via the same interview protocol. The interviews were semi-structured with initial prepared questions as indicated and asked of all interviewees. Follow-up questions were asked by the interviewer as needed for additional clarification, to expand on a topic, or to further explore a response. The final interview questions are included as Appendix B.