Criterios clave en el diseño de sitios web culturales

The data collection procedures and timeline are depicted in Figure 3.3. There were three main phases of data collection. At the beginning of the 2014 fall semester, I collected the EPT data and administered the self-assessment to ESL students, along with the motivation

questionnaire. In the second half of the 2014 fall semester, I conducted face-to-face semi- structured interviews with ESL students and acquired the TOEFL iBT and IELTS scores of the newly enrolled international students from the Registrar’s office. At the beginning of the 2015 spring semester, I conducted face-to-face semi-structured interviews with ESL instructors, content course instructors, and academic instructors. I also requested and collected the GPA data as well as the ESL course performance data following the approved procedures in the

Institutional Review Board (IRB) application. Detailed data collection procedures are described below.

Figure 3.3. Procedures and phases of data collection.

Fall 2014 Weeks 3-4

•Administer the self- assessment, as a part of a comprehensive

questionnaire containing items for motivational constructs

Fall 2014 Weeks 7-12

•Interview 8 ESL learners;

•Request for the TOEFL iBT/IELTS data from the Registrar’s office

Spring 2015 Weeks 1-3

•Interview 5 ESL instructors, 4 content course instructors, and 3 academic advisors; •Request for the GPA data. •Collect student performance

The self-assessment was administered in the format of an online questionnaire in weeks 3 and 4. The scheduling of the self-assessment administration was determined with a consideration that students were supposed to become familiar with the English language requirements in their content courses by then so that they would better be able to self-assess English proficiency with reference to their academic activities in their content courses. This decision was also informed by the findings from the pilot study, which showed that the timing of self-assessment administration might be a factor influencing the relationship between the EPT performance and the self-

assessment (Li, 2015). A weak correlation between the EPT and the self-assessment (.150 to .373) was reported in the pilot study, in which the self-assessment was administered in weeks six and seven of the spring 2014 semester. It is speculated that ESL students had made various gains in English proficiency after a one-and-half-month period of immersion at Iowa State University and this relatively long-period of immersion may fail to reflect ESL students’ initial English proficiency as measured by the EPT.

An email list of the test-takers of the fall 2014 EPT was requested from the EPT office and an invitation email was sent to all the EPT test-takers in weeks 3-4 in the 2014 fall semester through an emailing function on Qualtrics, a web-based survey tool (http://www.qualtrics.com). An electronic informed consent form was presented on the first page of the online questionnaire stating that student’s participation was voluntary. In the invitation email, the purpose of this study and its significance to the EPT and the ESL program were explained to the ESL students. Students were also notified that the participants in the study had a chance to win one of the three 10-dollar gift cards if they entered their email address in the questionnaire for a lottery. To improve the response rate to the questionnaire, I sent two reminder emails to the ESL students who had not completed the questionnaire at an interval of five days after the initial invitation

email. In addition, I sought help from the ESL instructors and asked them to introduce the questionnaire both orally in their classes and by email. In most cases, ESL students completed the questionnaire on their own outside classes. Time stamps and the time spent on each section of the questionnaire were automatically recorded by Qualtrics, which were used as a reference for data cleaning.

To ensure a good quality of self-assessment data, I manually cleaned and screened the data by identifying the respondents who spent little time on the self-assessment or showed disingenuous response patterns in the survey. As observed in the piloting of the questionnaire, it was estimated that five to eight minutes were needed to finish the self-assessment. If a student, for example, only used less than 4 minutes or responded to all the items using the same value, say, 1 or 6, I excluded this student from my data analysis. This screening process was intended to result in good quality of the data for analysis.

The semi-structured interviews with ESL students and ESL instructors were conducted in a neutral, unthreatening environment in the second half of the fall 2014 semester. The interviews with academic advisors, content course instructors, and ESL course instructors were conducted in the interviewee’s offices. All the interviews were conducted in English and were audio recorded with the permission of the interviewees (see Appendix B for a sample consent form). The length of the interview varied from 25 to 45 minutes. A member check was carried out later to ensure the accuracy of transcription and interpretation of interviewee’s responses (Glesne, 2011). In the member check, a sample of theme interpretation with interview excerpts was presented to the interviewees, if they were available. Feedback from the interviewees was incorporated in the data analysis and discussion.

The test score data, including ESL students’ TOEFL iBT or IELTS scores and their EPT scores, were requested from the Registrar’s office and the EPT office with approval from the Institutional Review Board (IRB) at the university. All the test performance data were de- identified for analysis after being matched with students’ questionnaire responses.

ESL students’ performance data in the ESL courses were collected as a source of evidence for progress in English proficiency. A pre- and post-test design was used in the ESL courses. In the reading and listening courses, students’ reading and listening scores on the EPT were recorded as a pre-test data set. The same in-class test was used at the end of the semester as part of the final exam and as a post-test data set. In the academic English writing classes,

students’ essays written on the first day of the class were used as a pre-test data set and their essays written for the final exam at the end of the semester were used as a post-test data set. Comparisons between the pre-test data and post-test data shed light on students’ progress in English learning.