ANALISIS ORIENTADOS A OBJETOS Diagramas CASOS DE USO

In the activity 2 is based on an activity of the Torrance test and how it was used in a previously published thesis (Shaheen, 2010). The scoring rubric of Torrance test was adjusted in both activities used by this research. The rubric for both activities can be found in the appendix 3d. The reason why this happened will be explained in this

163

section. For the same reason that some other variables, such as humour, were not included in order not to make the judgments subjective, the scoring rubric of Torrance was adjusted in order to be marked in a consistent way and to improve interrater reliability.

From this activity, the abstractness of the title and the premature closure of the students were assessed. The activity and the marking scheme (look appendix for the exact marking scheme used) are based on Torrance Test of Creativity Thinking (Torrance, Ball & Safter, 2008). However, I adjusted the marking scheme to be more precise. Furthermore, since the assessment of this activity is inter-subjective, and it can depend on the judgment of the assessor, I decided to use a smaller scale (0-2). I espouse that the more numbers this scale includes, the more fluctuation we could have in the scores. As this is a subjective evaluation, I decided to restrict the scale to avoid the variance and the potential arbitrariness.

The assessment of the resistance to premature closure can be challenging because it involves the evaluation of the shape of the picture drawn. To be more precise, the picture was scored with 0 if the figure is closed in one of the quickest ways, or the student wrote a letter(s) of the alphabet or number(s). This score was also given if the student closed the shape with one of the quickest ways and added details within the closed figure. The picture was scored with 1 when details were added outside of the enclosure. Finally, the picture was scored with 2 if there is no closure (the shape is open) or the shape was closed with the use of irregular lines as part of the picture. The way the marking scheme was phrased was different to the Torrance test. The aim was to make the marking scheme phrased in the simplest possible ways. Furthermore, the marking scheme in Torrance test (Torrance, Ball & Safter, 2008) created a big ambiguity when there was a reference to the drawing which score 2:

Closure is never completed is completed with irregular lines which form part of the picture rather than with straight lines or simple curved lines. (p.13)

The ambiguity concerned the first case. The test should clarify that the drawing is not completed but it is included in the drawing somehow. Otherwise, all the students who did not draw anything and left the item blank would have scored 2, since the drawing would be incomplete. Thus, I decided to change the phrasing in the marking scheme concerning this item.

164

This activity scored the resistance to premature closure and the abstractness of the title, which are two characteristics that creative people have. Initially, the peer raters and I tried to use exactly the Torrance Test scoring rubric. However, there were difficulties and weaknesses identified in this rubric. First, for the resistance to premature closure the test gives 0 for closed shapes, but 2 marks for a shape that is closed but considered to be closed with irregular lines. Therefore, what was considered irregular lines were sometimes questionable and this could cause a big variation between raters. If the lines were not considered irregular a student could score 0, whilst the same student could score 2 if marked by a different rater who considered the marks irregular. Torrance test might address this inconsistency with training of the raters and might mediate this problem because of the big number of pictures evaluated in this task. When this rubric was used for the piloting of this thesis, the inter-rater consistency was low. Therefore, a new rubric was created. Therefore, the marking related to combination of pictures was not applicable in this trial, because I included only one image given the concentration span of the age of the participating students.

Concerning the abstractness of the title, the scoring guidelines of Torrance test were also adjusted. Specifically, according to the guidelines the students score 0 if they state the obvious, the students score 1 mark if the title is simple descriptive with a modifier, such as ‘dancing cat’. However, the students score 2 marks if the title is imaginative and the modifier goes beyond concrete, such as ‘the dog named king’. Finally, an abstract title gets 3 marks. However, when there is a judgment between 1 and 2. First of all, the student might not have drawn something effectively and therefore something might not appear obvious from the picture. In that sense students who draw better might have been disadvantaged.

For the content validity to be achieved the activity should measure exactly what it states that it measures. The initial rubric gave additional marks for students who mention objects which do not exist. However, I adjusted the rubric in order to measure only the abstractness of the title independently of the picture. In this way the items of the assessment are kept independently. Furthermore, even though the importance of the students mentioning a title of an object that does not exist was recognised, this would still include many problems since it does not necessarily imply that it is more abstract. For fairness reasons, in the marking process it would be problematic to distinguish when students imagined an object or have seen it somewhere. For example, students might have seen an imaginary object to a TV programme or a book. It would have been

165

impossible for the researchers to identify all the possible influences and distinguish between demonstrating original imagination and reproducing of somebody else’s imaginative ideas. Thus, these were included in the same category.

Consequently, the abstract titles got the highest marks. Furthermore, if the title implied the existence of a story, the title was also considered as involving a high level of synthesis and organisation. This characteristic is associated with creativity and therefore the title got high grades. It might appear bizarre that the concrete title gets more points than the simplistic title, which is graded with 0, when the construct rewarded is abstractness of title. However, as Torrance, Ball and Safter (2008) reported, the abstractness of title includes initially the individuals’ ability to synthesise their thinking and in the highest level to capture the essence of information involved.

Even though for the 10% of the first data marking two more raters were used, it has to be clarified that one of the most important things was for me to be consistent. Since all the sheets were marked by me, it was important for me to consistent with myself (intra-rater reliability). Even if I was strict or lenient, since there were no other raters, the goal was my own consistency. To avoid conscious or unconscious bias based on whether the student is in the intervention or the comparison group, I marked the creativity activities in a blind way. When I was marking, I did not know whether a student was in the comparison or intervention group.