Colección y determinación

of administrative data for tracking.

A common limitation encountered by institutions in several countries is that regulations prevent the identification of students from disadvantaged social and ethnic backgrounds, with the result that they cannot be tracked. This has spurred the development of parallel voluntary surveys designed to collect student background information other than age, gender and prior education. New students receive questionnaires when they register, so that institutions can develop a clearer picture and target their information concerning student support services. The response rates are usually quite high, probably because these students trust and expect much from their institutions, and often do not know whether participation in such surveys is mandatory or not.

4.2.4 development of student services

The success of student-centred, flexible learning is highly dependent on the existence of effective student services. Tracking systems can boost them and ensure that they are efficient and targeted.

Tracking enables the identification of potentially at-risk student groups and thus an evaluation of the impact of the services required, for example by correlating the extent to which students engage in the life of the institution by using the library, attending classes, etc., with their retention and throughput rate. Tracking can also contribute to forward planning; for example surveys of applicants and new students may contribute to identifying the needs of future cohorts and improving the support offered. As demonstrated by the case studies that have formed the basis of this report, it was at times hard to draw the line between tracking progression paths, assessing the student experience and the actual support provided to students, as they go hand in hand. A number of universities pointed out that students at risk were often the most difficult to reach and that the impact of some remedial measures was limited. Few of the students most in need of preliminary remedial courses enrolled for them before beginning their main courses, which raises questions about their motivation and incentives. The immediate impact of measures such as mentoring is also very difficult to evaluate. Nevertheless, there are some signs that tracking results in more evidence-based student support services which could both make use of the data gathered and help to collect it.

4.2.5 Quality assurance

In some respects, tracking is closely related to both internal and external QA. The European Standards and Guidelines for Quality Assurance (ESGs) refer to the development of ‘institutional self-knowledge’ as “the starting point for effective quality assurance” and state the need for institutions to set up information systems:

Institutions should ensure that they collect, analyse and use relevant information for the effective management of their programmes of study and other activities. (p. 7 and p. 19)

and that, in addition to the internal use of information:

Institutions should regularly publish up-to-date, impartial and objective information, both quantitative and qualitative, about the programmes and awards they are offering. (p. 19)

The ESGs provide examples by referring to the following: student progression and success rates; employability of graduates; student satisfaction with their programmes; a profile of the student population; the need for the institution to establish its own key performance indicators; and the possibility of comparison internally or with other institutions (p. 19).

Tracking of students and graduates has become a requirement of external QA in some countries. It may either be prescribed as an activity (in particular, the tracking of graduates) or implicitly through the requirement to provide certain information and data. It could either – as is usually the case in accreditation schemes – comprise desirable procedures and standards (such as the time taken to obtain a degree, throughput rates, employment rates of graduates within a specified period after graduation, and student feedback on the quality of study programmes and facilities), or it could leave it to the institution to develop ways of demonstrating its quality.

54

Tracking, however, should not be limited to the fulfilment of external QA requirements. It should also be instrumental in internal QA, as it generates information and data useful for its systems and processes. Tracking can be instrumental in identifying the root causes of problems and the conditions for enhancing quality. They might include combined analysis of data on student entrance qualifications, course evaluations, and provision of a basis for discussion between academic staff, student services and QA staff. Similarly, tracking can help in analysing the relevance of institutional learning provision for employment.

The site visits revealed examples of different approaches to the linking of tracking with QA:

• Tracking was used to establish indicators for quality management. The quality management system ensures that the collected information is used in a systematic way and that follow-up measures are implemented. • Tracking student data (statistics on retention and throughput rates) is used as the basis for a quality

dialogue on teaching and learning between the institutional leadership and the leadership of the faculties. The information collected from different tracking activities was combined and used in a circular quality assurance system in which it is systematised and follow-up measures that can be monitored are introduced. The philosophy behind the quality dialogues is ‘How do you know?’ and ‘How do you act?’.

• Tracking at institutional level is linked with participation in national and international accreditation schemes, which requires (for example) the provision of information on the employment situation of graduates (including the transition period between graduation and entry into the labour market, and occupational data).

Thus while tracking can and should contribute to QA, it is not identical with it, and not necessarily linked explicitly to the institutional QA functions. Tracking is one of many instruments available to ensure and enhance quality. The overlap between the two functions depends upon:

• How the institution defines both tracking and QA. If both are seen as two dimensions of control or its opposite, enhancement, then there may be overlap between the two functions. If tracking is used for enhancement purposes and QA as a controlling process (or vice versa) then tracking would be used for different purposes in different parts of the institution.

• The scope of tracking as opposed to that of QA: several institutions are using tracking for QA purposes. Beyond providing evidence for internal or external QA reports, it appears to be closely linked to governance and strategic management, while QA is often focused on the evaluation of study programmes.

Interestingly, among the universities visited, there was no case in which the QA office was in charge of tracking. However this may have to do with the different professional abilities needed for the two functions. Also significant is the fact that, at many institutions, QA has become a task for the entire institution, with the QA office in a supporting and coordinating role including its activity in preparing for external QA.

Regardless of how tracking is approached, institutions need to establish explicit feedback loops, which would ensure systematic use of the results of tracking and their contextualisation. The latter is important with regard to overall institutional strategic goals, but also in preventing tracking measures from leading to simplistic conclusions, which could do more harm than good.

55