Factibilidad técnica de la central generadora

Research would be worthless if it was invalid or unreliable. It is therefore necessary to talk about the validity and reliability of this study. According to Cohen, Manion and Morrison (2000), validity in research means that instruments measure what they are supposed to measure. Qualitative validity can be achieved through honesty on the part of the researcher, the depth and richness of the data and the suitability of the subjects. On the other hand, quantitative validity can be achieved through choosing the study sample carefully, using appropriate instruments and selecting appropriate statistical analyses for the data.

The decision to base the study in the English language department at Al-Qassim University was instrumental in ensuring the validity of the study. This is because a course in teaching writing to second-year students which included learning writing skills through stages and activities: pre-writing, drafting, revising and editing, had already been set up at the

123

department. The setting was thus ideal for the aim of this research, which was to determine the effectiveness of collaborative learning in improving the process approach to writing.

Regarding the use of appropriate instruments, the pre-test and post-test essays were assessed using the scales of Paulus’s rubric (1999). All the scores of the participants in both the experimental CL group and the control TL group were judged and rated by two near- native expert teachers. The researcher chose two judges or markers and a third to act as adjudicator if there was no correlation between the first and second markers.

The details of all markers’ ratings are given in Appendix F. The judges used the essay-scoring rubric from Paulus (1999). The scale went from 1 as the lowest level to 10 as the highest, and the two judges gave both total and analytical scores (see Appendix D). In order to test whether there was a correlation between the first and second markers; Cronbach’s alpha was used to calculate inter-rater reliability between the judges. Cronbach’s alpha measures intra-class correlation and is considered to be an indicator of internal consistency (Howell, 2002; Cortina, 1994, cited in Larson-Hall, 2010). It is therefore important to establish a positive correlation in order to ensure consistency between the two judges.

In order to achieve reliability, the majority of the judges’ ratings should vary in a similar fashion according to the participants they are judging. For example, if judge A gives participant 1 a high score and participant 2 a low score, judge B should also give participants 1 and 2 similar scores. According to Larson-Hall, (2010), variations in the sample are recommended (e.g., student A got 20, student B got 25), whereas any variation between the judges will make the rating less reliable. Larson-Hall (2010) also states ‘If judges are consistent then there shouldn’t be too much variation in these numbers. However, if there were a certain judge whose data change Cronbach drastically you might consider throwing out that judge’s scores’ (p. 173). Consistency between the judges would indicate small

124

variations in students’ marks, which is considered a positive correlation. In this case, a mean score for each student would be recalculated from the combined scores given by markers 1 and 2.

It is important to have validity and reliability among the judges. Huot (1990) refers to ‘The value of the judgment given by a rater (validity) and the ability of the raters to agree (reliability)’. Raters must judge an essay according to similar features if they are to agree with each other. The researcher gave all the raters the same rubric and trained them how to use it effectively in order to obtain valid and reliable results.

In terms of the reliability of both the questionnaire and the interviews, the researcher discussed with the students the procedure involved in completing the questionnaires and the importance of doing so honestly and accurately in order to enable him to collect valid and reliable data. With regard to the interview instrument, the researcher chose four students at random from the treatment group to represent their classmates. Referring to the previous term’s writing exams, the researcher selected student A to represent any students who had obtained 50-60 marks; student B for any students who had got 60-70; student C to represent any students who had scored 70-80; and student D to represent any students who had got over 80.

The trustworthiness of results obtained from instruments or tests can lead to four types of validity: content validity, which is a measure of how effectively the items represent other items. In the current study, in order to ensure content validity, the assessment instrument had to include all the procedures necessary for measuring writing ability. The second type of validity is concurrent validity, which is a measure of how accurately the researcher is able to correlate one test with another. Predictive validity is the third type, and is a measure of how effectively a test or instrument meets a criterion. It is considered an important kind of validity in placement tests where the raters are able to predict the success

125

that students will achieve in any specific English language course. Lastly, construct validity is a measure of how well a test assesses some underlying construct (Huot, 1990; Salkind, 2000).

In addition to the types of validity described above, the researcher in this study used an experimental design that was evaluated by the two criteria of internal and external validity. He selected second-year students as the sample for this study to represent all students of the English language department with the aim of achieving a high degree of generalizability. Ensuring that this study could be applied in different situations with similar characteristics was one of the main goals of the researcher, since this would give the research external validity and mean that the findings would represent all ESL students in the world, thus achieving the goal of generalizability.

Regarding the reliability of the study, Cohen, Manion and Morrison (2000) mention three principles of reliability relevant to quantitative research: stability, which measures consistency through both time and across similar samples; equivalence, which can be achieved either by using similar forms of a test or instrument or by inter-rater reliability, when two researchers are involved in the research and different independent judges agree that both researchers entered data in a correct and similar way, and internal consistency, in which the tests or instruments are required to be applied twice. The researcher tried to make the research as stable as possible in order to achieve reliability. He selected two groups with similar characteristics (i.e., level of classes and age) in order to ensure the consistency and stability of the results.