Institucionalidad del Sector

A multidimensional scaling analysis (MDS) and a hierarchical cluster anal- ysis (HCA) were computed based on the country means for each of the questionnaire items. Countries were grouped based on the similarity of their responses. Differences in teaching practices across countries were identified along one main dimension. The correlations between the di- mensions of teaching practices and PISA progress scores in reading, mathematics and science were computed.

See Appendix 1 for further details on analysis.

5.8 Results

The outcome of the MDS can be observed in Figure 1. Two dimensions, horizontal and vertical, are displayed, with the Nordic countries appearing at the upper left extremity.

5.8.1 Country clusters

The model differentiates broadly between Eastern European countries (right) and Western European countries on the left, indicating that re- sponses to the questionnaires reflect, in part, the geographical positions of the countries under study. This indicates that the major cultural differ- ences in Europe are observed between Eastern and Western Europe. In- terestingly, within the Eastern European group, Russia (RUS) and Georgia (GEO) are in close proximity, perhaps not surprisingly because these two countries were parts of the former Soviet Union. Similarly, Slovakia (SVK) and the Czech Republic (CZE) are relatively close on the graph, perhaps reflecting similarities between these countries dating from the time when they formed a single country.

The four Nordic countries included in the study (FIN, NOR, DEN and SWE) are relatively close together in the upper left corner of the graph.11 The two Germanic countries, Germany (GER) and Austria (AUT), are also in close proximity and adjacent to the Nordic countries. Towards the low- er left part, a set of largely Anglo-Saxon countries are grouped together, including Ireland (IRL); Malta (MLT); Australia (AUS); Québec (CQU), and Northern-Ireland (NIR). In the lower right part are Italy (ITA) and Portu- gal (PRT), with similar scores on the Dimension 1, but with Spain (ESP) being situated a bit further up on Dimension 2.

The hierarchical cluster analysis (HCA) based on the country scores on the variables, assists in determining the groupings (Figure 2). It is largely in line with what has been described above, with the exceptions that Spain is clustered with Slovenia and Italy and Portugal are clustered with Romania.

5.8.2 Interpreting dimensions that differentiate

between country groups

The dimensions will now be interpreted on the basis of how they correlate with the country means for each of the variables used to generate the MDS solution. Each dimension will be interpreted by exploring the correlations between the countries’ coordinates on the dimension and the individual question items. It is not practical to present all the correlation coefficients. Instead, the interpretation of the dimensions is based on significant correla- tions and summarising and paraphrasing the content of the items that cor- relate significantly with the dimensions. Pearson correlations are used.

The horizontal and vertical dimensions were interpreted separately. However, their meanings were similar in that both seem to reflect differ- ent degrees of engagement. This horizontal dimension correlates signifi- cantly with two progress measures in PISA (reading and mathematics) and less well with science literacy. The vertical dimension, on the other hand, correlates with progress in science. This indicates that the high cor-

──────────────────────────

11 _{Iceland did not participate in PIRLS and TIMSS in 2011. Judging from a similar analysis of PIRLS data}

relations with progress may be achieved by correlating progress with a third axis, going diagonally from the bottom right to the top left.12

Because the dimensions largely differentiate between Eastern Europe- an and Western Eurpean coutries, as well as other familiar country groups (Anglo-Saxon, Nordic and Germanic), the terminology normally used when decribing correlations (e.g., a high score for x is associated with a high score for y) will sometimes be put aside, and descriptions will be made in terms of differences between groups of countries, a strategy that would normally only be used when the statistical methods used to com- pare groups are applied (e.g., ANOVA).

The description of the dimensions largely reflects the division of the ques- tionnaires into four sections: general, reading, mathematics and science.

By correlating the country scores for each question item with the coun- tries’ coordinates on the horizontal dimension, we are able to identify the characteristics that tend to be higher in countries on the left compared with countries on the right on the dimension. These characteristics will now be described in some detail. We will then do the same for the vertical dimension. Indications of which sections of the questionnaires the items at hand come from, should readers want to look them up on the TIMSS and PIRLS websites, appear in brackets.

The horizontal dimension (Dimension 1) will be interpreted first.

Teachers’ education

Regarding the teachers’ formal education in teaching reading, in countries on the right, more emphasis on studying the language itself, the pedagogy of teaching reading, educational psychology, remedial reading, reading theory, language learning and assessment methods in reading is reported by teachers (R20). However, teachers on the left end of the dimension

──────────────────────────

12_{“Orientation of axes. As in factor analysis, the actual orientation of the axes in the final solution is arbi-}

trary.” For example, although a map can be rotated in any way we want, the distances between cities will remain the same. “Thus, the final orientation of axes in the plane or space is mostly the result of a subjec- tive decision by the researcher, who will choose an orientation that can be most easily explained.” (…) (W)e could have chosen an orientation of the axes other than north/south and east/west; however, that orientation is the most convenient because it “makes the most sense” (i.e., it is easily interpretable). http://www.statsoft.com/Textbook/Multidimensional-Scaling.

have placed a relatively greater emphasis on special education with regard to teaching reading. This is in line with their greater tendency to send pupils with reading problems to specialised professionals (R18).

Better prepared to teach?

Countries on the right part of the dimension report having greater confi-

dence (S2, M2) in performing various teaching tasks. They undertake

more hours of professional development (R21) and have read children’s books more often as part of professional development (R22). Conversely, teachers on the left feel more “frustrated” (G11).

Teachers’ contact with one another

In countries on the right, teachers collaborate more with other teachers. They plan and work together and visit one another’s classes to learn from one another (G10).

Assessment and homework

Teachers in countries on the right end of the dimension accord greater im- portance to assessment (R19, S9) in both maths and science. They place great- er emphasis on homework in reading, maths and science, expecting students to do more of it and monitoring whether it has been completed or not more frequently. They correct the homework and discuss it in class more frequent- ly. This is the case in all three subjects: reading, maths and science.

More front-loaded curriculum?

A few topics in science and maths are taught earlier in the countries on the right. This was determined using a scale that assesses whether individual topics were taught before this year, were taught during this year or have not yet been introduced.

Use of computers and books

Computers are more accessible in the countries on the left, but teachers in countries on the right end report more frequent computer use in science lessons (S5). In a similar vein, they ask their students to read or look some- thing up on the computer more often than countries further to the left. They also report requesting the use of textbooks on the part of their students more frequently.

However, in the countries on the left, greater emphasis can be seen in various questionnaire sections regarding computer use. They have more computers, use software more frequently, have greater internet access and use computers more frequently in teaching preparation and class- room instruction than countries on the right.

More student engagement

Teachers of maths, science and reading were asked how often they ask pupils to perform various learning tasks. Again, teachers in the countries on the right end of the dimension present a similar picture in terms of how they conduct their teaching of maths, science or reading (S3, M3, R9 and R10). Science teachers, for example, in the countries on the right, more frequently ask pupils to relate what they are learning to their daily lives, give explanations of what they are studying, memorize facts and principles and observe natural phenomena such as the weather and de- scribe what they see. They also more frequently ask students to watch a demonstration and read textbooks.

The pictures of mathematics and reading lessons are simliar. In math- ematics, teachers more often ask pupils to explain their answers; relate what they are learning to their daily lives; memorize rules, procedures and facts; and work on problems together as a class, with direct guidance from the teacher.

In reading, teachers in countries on the right of the dimension also ask pupils to perform certain tasks more frequently in order to help them develop reading comprehension skills and strategies, e.g., locate infor- mation within text, identify the main ideas of what is read, explain or sup- port their understanding of what they have read, compare what they have read with their own experiences or other things they have read, make predictions about what will happen next in the text, make generalizations based on what they have read, and describe the style and structure of a text and the author’s intention (R9). They also more frequently ask them to write something about what they have read, answer oral questions or talk among themselves about what they have read. In reading lessons, teachers in countries on the right more frequently ask students to read aloud, teach them strategies to decode sounds and words, teach new vo- cabulary sytematically and teach skimming or scanning strategies (R8).

Countries on the left are, however, more likely to give students time to read books of their own choosing and give them longer books to read. Also, pupils in these countries have more children’s books available for their perusal (R6). Overall, teachers on the left complain less about a lack of teaching materials (G8).

In countries on the right, teachers’ interactions with their students are also more engaging in the sense that they more frequently relate the les- sons to the students’ daily lives, encourage them to improve their perfor- mance, praise them, bring interesting materials to class and summarize what they should have learned from the lesson (G15).

Contact with parents

Teachers in countries on the right side of the dimension are in greater contact with parents, more frequently meet or talk individually with par- ents to discuss the learning progress of their offspring and more frequent- ly send home progress reports (G17).

Tests

In all three subjects in countries on the right, great emphasis was placed on taking a written test or quiz about what the students had studied or read.

When students fall behind

When students fall behind in reading, teachers in countries on the right tend to spend more time with them, while refering them to specialised professionals is more the rule in countries on the left (R18). Teachers in countries in Western Europe more frequently complain that students lack sleep or disrupt classes.

Gender and length of service

A greater percentage of teachers on the left are males (G2). Teachers on the right had taught for longer and were older.

The vertical dimension (Dimension 2) is similar to the horizontal dimen- sion in the sense that being lower on the dimension is associated with greater engagement of various kinds, just like the right side of the horizon- tal dimension. Fewer items correlate with it, however. Teachers in countries on the lower end tend to perform more professional development, report a higher level of understanding of their schools’ curricular goals and spend

more time teaching mathematics overall. They more frequently ask stu- dents to read aloud and teach strategies for decoding sounds. Also, they make greater use of some of the reading comprehension strategies, as seen in the horizontal dimension. In mathematics, pupils are more frequently asked to memorize rules and explore mathematics principles and concepts as part of computer activities. They are also asked to do more homework than students in countries toward the top.

Teachers in countries on the bottom of the dimension have higher ex- pectations in terms of student achievement and encourage students to improve their performance more frequently. Similarly, pupils are reported to have a greater desire to do well, behave in an orderly manner, be re- spectful of teachers and have a high regard for school property than those in countries on the top of the dimension (G6, G7).

Teachers in countries on the top of the vertical dimension report that they have fewer pupils in class. They share what they have learned about their teaching more often, and they have greater access to teacher aid when a student has difficulties with reading.

5.8.3 Teacher practices and progress as measured via

PISA

The country scores on the two dimensions were correlated with three measures of progress in PISA. The correlation coefficients (Table 1) indi- cate that high scores on the horizontal dimension (engagement) are asso- ciated with PISA progress in reading and mathematics. A high score on the dimension indicates low levels of engagement, countries on the left of the MDS solution (the Western European countries), i.e., those showing rela- tively little engagement in teaching activities, are making less progress in reading and mathematics. A low score on the vertical dimension is associ- ated with more progress in science.

Table 1: Pearson correlations between the horizontal and vertical dimensions and the three measures of PISA progress (reading, mathemics and science).

Horizontal dimension Vertical dimension

Progress_PISA_Reading 0.606* -0.075 p=0.013 p=0.782 n=16 n=16 Progress_PISA_Mathematics 0.573* -0.407 p=0.026 p=0.132 n=15 n=15 Progress_PISA_Science 0.410 -0.530* p=0.073 p=0.016 n=20 n=20

*significant at the 0.05 level (2-tailed)