Consistencia

The ranking phenomenon for universities and other higher education insti- tutions has expanded rapidly since the early years of the present century, and there are currently a large number of rankings with various focuses in the hig- her education sphere. There has been keen criticism of most of these ranking systems but, in the current debate, there is hardly anyone who would advocate their elimination. The most common view of the ranking of universities and other higher education institutions is, instead, that “they are here to stay, and what can we do about it?” And this approach also characterises most of the trends in the development of ranking systems.

Greater interest in quality aspects

On the one hand, there is a trend that is in opposition to the fundamental prin- ciples on which rankings are based – the presidents of several U.S. and Cana- dian higher education institutions no longer supply information to influential 63. Hence, a higher proportion state that they were influenced by rankings than those who

claim that they used rankings as a source of information. This may be because the questions have been misunderstood. But this is not necessarily the case, since it is possible to obtain information about ranking results from indirect sources. The questions are also posed in different ways, and this may explain the different responses.

ranking designers (U.S. News & World Report and Macleans, in Canada). This means that such institutions will no longer be represented in the most prominent ranking lists – and this clearly has an impact on their visibility and publicity, and may affect future student recruitment. In this case, however, irritation about the way rankings are carried out appears to have been the over- riding factor. Only time will show if this means that rankings will improve.64

On the other hand, there is widespread interest in finding out more about ranking. There is considerable interest in research on, and the evaluation of, different ranking systems (this report is only one of several similar attempts). This interest exists on the part of higher education institutions, governments, public authorities, national and international organisations, researchers, stu- dents and so on. Quality issues in ranking are a hot topic. Many conferences are held in this area, and there is a growing volume of literature in this field.

One indication of the topicality of the ranking quality issue is the Unesco- Cepes observer function that has been recently established to monitor whether rankings comply with the principles laid down in Berlin in 2006 as a result of the formation of the International Ranking Group in 2004. The 16 “Berlin Principles” cover the purposes, objectives and target groups for ranking, the design and weighting of indicators, the collection and processing of data, and the presentation of ranking results. It is hoped that the formulation of these Principles will result in greater awareness of methodological ranking issues, and hence to improvement and refinement of the ranking methods applied. The formation of an observer group to monitor rankings has revived the focus on the Berlin Principles, although there are no sanctions that can be invoked against ranking designers who fail to comply with these Principles.65

The Institute for Higher Education Policy in Washington has recently esta- blished a “policy centre” to monitor ranking trends and specifically to gather information, in the form of a “Ranking Clearing House”. This is yet another sign of increased interest in ranking and research in this area.66

Much of the criticism of ranking lists involves the biased selection of indi- cators. The question is raised of whether simple measures for resources and results are actually capable of covering the quality aspects of higher education to any great extent. Projects are also under way which attempt to describe the outcome of higher education in a more sophisticated form than ranking. The OECD Assessment of Higher Education Learning Outcomes (AHELO) is a project of this nature, in which the possibilities of assessing the quality of higher education in the form of knowledge tests similar to those employed in 64. Provan & Abercromby (2000). Swedish National Agency for Higher Education Interna-Provan & Abercromby (2000). Swedish National Agency for Higher Education Interna-

tional News Letter 2006: No. 22 and www.insidehihered.com/news/2008/09/26/colleges- peaks

65. University World News, 1 June 2008 and www.ireg-observatory.org. See www.ihep.org/ Organization/Press/Berlin_Principles_Release.pdf for a full account of the 16 rather ambi- tious Berlin Principles. See also Merisotis & Leegwater (2007). McCormick (2008) applies the Berlin Principles for classification rather than ranking.

66. Alisa F. Cunningham, address to the IMHE General Conference on 8–10 September 2008. See also www.ihep.org/Resarch/rankingsystemsclearinghouse.cfm.

undergraduate education (e.g. PISA tests) are being investigated. This project is still in its early stages, but a pilot study may be initiated in certain areas and countries in 2009.67

A European ranking market?

A specific European trend in the ranking field may be anticipated. Valerie Pécresse, the French Minister of Education, has notified that she intends to raise the question of a European ranking or classification system at a meeting of the ministers of education of the EU countries in November of 2008. One reason is that it is assumed that the Academic Ranking of World-class Univer- sities conducted by the Jiao-Tong University in Shanghai, which is possibly the most well-known ranking system, gives a less favourable rating to European Universities (particularly non-English speaking institutions).68

The German Centrum für Hochschuleentwicklung (CHE) has declared that it is interested in extending its system for ranking of Master’s program- mes to the whole of Europe as a result of the Bologna process and the possible subsequent increase in mobility in Europe in the higher education field. In point of fact, the CHE has already extended its German rankings by including Austrian, Swiss, Belgian and Dutch institutions. The CHE has also implemen- ted a ranking of “excellent” Master’s programmes in Europe in the natural science and mathematics areas.69 _{In other words, there are plans to extend this}

operation into other academic fields.70

Multidimensional and interactive rankings

The German CHE organisation applies a slightly different ranking approach, as described in more detail in the next chapter. This ranking system is not based on a joint weighting of indicators, but takes one indicator at a time. The CHE describes this approach as “multidimensional” ranking. The ranking results are published in Die Zeit once a year – for a number of different indi- cators for several different education programmes in Germany. It is, however, possible – all the year round – to refer to a website and personally select the indicators and the programmes for which information is required.71

This type of multidimensional and interactive ranking (if it is to be termed ranking) is increasing all along the line. Several British, American and Cana- dian newspapers are currently offering possibilities of studying other indica- tors, subject-classified lists, etc. via websites, in addition to their traditional ranking lists. The results often contain lists with the same methodological pro-

67. www.oecd.org/document/22/0,3343,en_ 2649_35961291_40624662_1_1_1_11,00.html 68. EurActiv.com, 1 September 2008.

69. Berghoff, Brandenburg and Möller-Böling (2007). 70. Berghoff, Brandenburg and Möller-Böling (2008).

71. See www.daad.de/deutschland/hochshulen/hochschulranking/06543.en.html for the English- language version.

blems as the main rankings, but in some instances , as in the German case, the user can personally select the indicators and weightings applied.

The purpose of multidimensional rankings is often explicitly claimed to be to give potential students information about higher education which they themselves consider to be relevant. Such rankings (to the extent that they can be regarded as rankings) are broken down in terms of subjects or education programmes, and this further enhances their relevance from a student perspec- tive. The breaking-down of information, and the categorisation of subjects or programmes presents problems, however, and the results are not always clear and unambiguous. This type of information is even more difficult to handle in international comparisons, since each country has its own classification sys- tem, which is not always easy to compare across national frontiers.

Survey of existing ranking