Perspectiva cognitivas

This section describes the analytical strategy and statistical analyses that were applied in addressing each of the research questions.

Research question 1: What are the LMS components most used by online language instructors?

Sub-questions:

• Is there a significant difference across LMSs?

• Is there a significant difference across languages taught?

• Is there a significant difference between novice and experienced online instructors?

The first research question aimed to analyze which factors influence the use of LMS components. Therefore, all data were first analyzed using descriptive statistics to provide simple summaries about the sample and the measures, and to detect general tendencies.

In order to determine to what extent different LMSs, languages taught and levels of online teaching expertise influenced the frequency of use of each one of the LMS components, a series of binary logistic regressions were conducted. The objective was to establish whether or not there was any systematic relationship between each one of the explanatory factors (LMS, language and expertise) and the probability of a higher use frequency of LMS components. The frequency of use of fifteen different LMS

components, such as discussion boards, student document sharing or content modules, were considered as the dependent variables.

Dependent variables were ordinal variables, which indicate how often each one of the LMS components was used by the participants. The data were originally codified in five categories, which ranged from “very frequently” to “never.” Then, due to the relatively small size of the dataset (97 responses), they were further reduced into binary categories. The LMS and Language Taught factors were re-categorized to reduce the number of categories (and therefore, the number of cells with zero frequencies)—that is, the combination of variables that were not represented.

Research question 2: What is the relationship between the use of different LMS components and the learning activities provided for students?

Sub-questions:

• Is there a significant difference across LMSs?

• Is there a significant difference across languages taught?

• Is there a significant difference between novice and experienced online instructors?

The second research question for this study focused on determining whether or not there is a systematic relationship between the use of different LMS components and the learning activities provided for students. Once again, all participants’ responses were first analyzed using descriptive statistics to detect general tendencies.

In order to determine whether or not there is a relation between the use of different LMS components and the learning activities provided for students, as well as a significant difference across LMSs, languages taught, and/or the level of instructors’ online teaching experience, a series of contingency tables and likelihood tests were performed. Adjusted standardized residuals were also computed to determine which categories (cells) were the major contributors to the significant associations. The independent variables were (as in the previous question) the LMS used, the language taught and the online teaching experience of the instructor. The dependent variables were different learning activities computed as binary variables—namely, the

implementation (1) or not (0) of a learning activity by the instructor during the course.

Research question 3: What is the relationship between the use of different LMS components and pedagogical approaches?

Sub-questions:

• Is there a significant difference across LMSs?

• Is there a significant difference across languages taught?

• Is there a significant difference between novice and experienced online instructors?

The goal of the third research question was to determine whether or not there was a relationship between the use of different LMS components and pedagogical approaches—that is, do pedagogical preferences significantly influence the preference of use of some LMS components over others? As in the previous section, all

participants’ responses were first analyzed using descriptive statistics to detect tendencies.

Participants were asked to indicate if they agreed, disagreed or neither agreed nor disagreed with twelve statements regarding teaching methods, so as to gauge their stance toward three general pedagogical approaches in the teaching of foreign

languages. Four statements were developed to accord with behaviorist language learning principles, four with cognitivist language learning principles, and four with constructivist language learning principles. A Cronbach’s alpha test was conducted to check the internal consistency of each pedagogical group measurement scale (4 items each). All three question groups obtained an alpha value higher than 0.7. An average score was computed for each statement in order to gauge each participant’s general preference toward each pedagogical orientation.

In order to determine to what extent a preference for a pedagogical approach influences the frequency of use of each one of the LMS components, a series of binary logistic regressions were conducted as with research question 1. The objective was to establish whether or not there was any systematic relationship between the explanatory variables (the average score of behaviorist, cognitivist and constructivist statements) and the probability of a higher use frequency of LMS components. The explanatory variables were the average score of each pedagogical approach and the dependent variables— the frequency of use of each one of the fifteen LMS components—were reduced to binary categories: frequently used (1) and never or very low used (0).

Research question 4: How is teaching online through a LMS related to language instructors’ teaching practices and perceived ability to enact their pedagogical preferences?

Quantitative sub-question:

• To what extent do online teaching practices accord with instructors’ pedagogical principles?

The last research question has three sub-questions. The first questions required a quantitative analysis, which was complemented with qualitative examination in order to enhance a deeper understanding of the numerical results (Creswell and Plano Clark, 2007, p. 91).

As in previous sections, descriptive statistics were used in order to present distributions and trends of how often instructors believe they have to yield their teaching practices and pedagogical preferences to the capabilities of the LMS in their courses, as well as to identify the teaching practices that online language instructors commonly use in their courses.

Gamma tests of association were conducted to examine if there was a relationship between participants’ expressed pedagogical preferences (Behaviorist, Cognitivist and Constructivist) and their perceptions about the necessity to yield some of their pedagogical preferences in online courses. Gamma tests were also carried out to measure the extent to which online teaching practices accorded with instructors’

pedagogical principles. In a second step, ordinal regressions were done in order to infer the association between participants’ pedagogical preferences and the necessity to yield among participants.

Qualitative sub-questions:

• What do online language instructors perceive to be the major sacrifices they make pedagogically when teaching online?

• What do online language instructors perceive to be the major pedagogical gains when teaching online?

Analysis for the last two sub-questions of research question 4 followed a

qualitative analysis, which involved an inductive coding of data. Participants were asked to indicate what they perceived to be the major language-learning pedagogical

limitations and gains when teaching languages online through a LMS. They were asked to mention at least one issue and a maximum of three. Additionally, they were required to indicate what they perceived to be the major challenges.

Participants’ open-ended responses were qualitatively analyzed using inductive coding. The first step was to read carefully through all responses to get a general perspective; then a second reading was performed, and a preliminary coding was created de novo. A series of short phrases that represented the key attributes of the unique responses were created. The main objective of carrying out an inductive coding data analysis was to limit the possibility of forcing a preconceived set of categories on the data, based on the researcher’s experiences and existing literature. The preliminary coding was reviewed by performing a constant comparison process of moving back and forth from the codes to the data. The codes that emerged from this first iterative data analysis were then grouped into categories.

The second step of the qualitative analysis was the development of two preliminary coding frameworks: one for the participants’ perceptions of pedagogical limitations when teaching languages online, and the other for the perceptions on the pedagogical gains. Subsequently, an “intercoder agreement” procedure (Miles and Huberman, 1994 cited in Creswell and Plano Clark, 2007, p. 212) was implemented. Participants’ texts were coded by a second coder using the preliminary coding

frameworks; then, both coders’ codings were compared to determine where they agreed and disagreed, and thus, where the coding categories needed to be made more explicit. When different codes were assigned to a segment of data, a discussion about the reasons for coding a text in a certain way took place, and an agreement was pursued. After a coding comparison, discussion and analysis, the coding frameworks were

redefined and participants’ texts were coded a second time. The process of moving back and forth from codes to data was finished until exhaustiveness was reached; that is, until no new codes emerged and all data were coded.

Finally, all categories were compared with each other and consolidated into broader thematic constructs (second coding process suggested by Saldaña, 2009). As the categories of both frameworks showed some similarities, the researcher decided to organize them in a parallel way and encompass them into a single thematic scheme. In very broad terms, the codifying process followed a streamlined scheme (modified from Saldaña, 2009, p.12) as illustrated in Figure 3.2.

Figure 3-2 A Streamlined Codes-to-Thematic Scheme for Qualitative Analysis

Real Abstract Code Code Code Category Code Code Code Category Themes/Concepts Code Code Code Category Code Code Code Category Thematic Scheme L im it a ti o n s G a in s Themes/Concepts