Estructura urbana

4.7.1 Definition and Purpose of Evaluation

Evaluation is generally accepted as an integral part of the educational

system, curriculum, and quality improvement (Goldie 2006). Evaluation is the process of obtaining, analysing and interpreting information in order to

assess status, strengths, weaknesses, and merit of an educational

programme (Dornan et al. 2011; Yarbrough et al. 2011). Evaluation is used for several purposes: curriculum evaluation, accreditation, feedback on course/teaching, improving educational content and methods, supporting faculty development and promotion, and demonstrating accountability of the educational programme to the public (Harden and Crosby 2000; Kogan and Shea 2007). It can be summarised that evaluation is a process of

understanding the quality of an educational programme and educational process (e.g. teaching/learning/assessment) within the programme.

The focus of evaluation can be classified differently; for instance, evaluation can focus on an educational programme, its components, and stakeholders (Goldie 2006), or focus on the determination of the quality: utility, feasibility, propriety, accuracy, and accountability of a programme (Yarbrough et al. 2011). However, the focus of evaluation needs to fit with the purpose of evaluation (Wall 2010). Similarly to assessment, evaluation can be either formative (for improvement) or summative (for making judgment) (Firmstone et al. 2010). The common purposes of evaluation are to appraise the quality of teaching or the quality of an educational programme/curriculum; and to provide recording of a change of practice to benefit the public (Goldie 2006).

4.7.2 Evaluation of Teaching

The aims of teaching evaluation are to improve the quality of teaching and to ensure that students receive the best teaching which enable them to

effectively develop learning (Snell et al. 2000) and to demonstrate accountability to the public (Ory 2000). The perspectives of teaching evaluation could include: personality of teacher, teaching competences, discipline knowledge, and professionalism; these can also include input from teachers, students, service users (i.e. patients), and the institution (Jones 1989; Snell et al. 2000). However, it should be remembered that the more perspectives the evaluation covers the more resources and collaboration from stakeholders are required. It is important to utilise the most appropriate cost-effective strategy for evaluating teaching.

A number of methods have been employed to evaluate teaching; this includes self-rating, student rating, peer reviews of teaching, rating from colleagues, classroom visitation (Kulik and McKeachie 1975; Snell et al. 2000). Each method has its own strengths and weaknesses that need consideration. For example, student rating is found as valid, reliable, relatively free from bias, and covers a range of measurement perspectives (Marsh 2007; Clayson 2009); however, the data are only gathered from the student viewpoint which may not fully reflect every aspect of teaching. As for peer review of teaching, this method provides data relating to teaching competences and professional issues; it is also less judgemental and more constructive (Fernandez and Yu 2007). However, lack of standards for peer evaluation and problems with trustworthiness (unpredictable validity,

reliability, and biases) are some examples of its disadvantages (Chism 1999).

Teaching is just one component of the UG-DentalEduc (see Chapter 2). Good teaching evaluation results and high teaching quality may not imply that the whole quality of UG-curriculum is good/high nor reflect that the goal of the curriculum is successfully achieved.

4.7.3 Programme Evaluation

A wide range of evaluation models have been developed to evaluate the quality of an educational programme or a curriculum. These models can be classified into several orientations based on the focus of evaluation:

objective, management, consumer, expertise, adversary, and participant (Goldie 2006). Regardless of the orientation, one evaluation model, which is generally used and proven as useful in evaluating educational programmes in many health professional disciplines, is the CIPP model (Chavasse 1994; Leahy et al. 2009; Mohebbi et al. 2011). The model covers four areas of an educational programme: Context, Input, Process, and Product (Stufflebeam and Shinkfield 2007). However, arguably, this model only focuses on

components and processes within an educational programme; it may or may not reflect whether the programme goals/outcomes have been achieved.

An evaluation model which is widely used for evaluating ‘outcomes’ of

learners and of the programme is Kirkpatrick’s Model (Kirkpatrick 1975; Wall 2010; Frye and Hemmer 2012). Regarding this model, an educational

programme can be evaluated at five levels: participation, reaction, learning, performance, and outcomes (improvement of patient care). Many evaluations aim at the first and second because they are less complex and are easy to measure. In order to evaluate programme outcomes relating to the students, the third or fourth level would be more appropriate as they directly reflect student achievement. However, if an aim of dental education is to provide competent dental professionals to serve the society, the evaluation needs to focus on the highest level (i.e. how dental education improves people’s oral health); although it is difficult to measure.

4.7.4 Implications for Dental Educators

Evaluation is fundamental for quality improvement (Hobson et al. 2008); however, it is also problematic. Teaching evaluation helps educators to improve their teaching and assessment strategies but it is not necessarily

true that high teaching quality relates to quality of the whole educational programme. Other components of an educational programme also need consideration. An evaluation model such as the CIPP model can provide data covering a rage of components of the programme; however, it could not indicate whether the programme achieve the outcomes/goals. The outcome model (e.g. Kirpatrick’s Model) can demonstrate the programme achievement at the outcome level; though, it is difficult to measure the outcome directly due to the complexity. Moreover, student achievement could be a result of self-directed learning rather than high teaching quality. Thus, the question raised from this situation is what we (educators) should evaluate: teaching, programme components, or outcomes.

The above situation can probably explain why, although evaluation is crucial for identifying and improving quality of teaching and the UG-curriculum educators (Hobson et al. 2008), Hand (2006) found that evaluation is not perceived as an important issue for dental educators regardless of their roles. Educators may not recognise the real benefits of evaluation; instead, they possibly perceive evaluation as a threat. Similar to curriculum, evaluation is involved at different levels of an educational programme. If the goal of

evaluation emphasises the quality of teaching and learning, educators would appreciate evaluation as relevant to their teaching performance. However, if the aim of evaluation is to improve the quality of the programme and be utilised at an administrative level, educators might not perceive evaluation as relevant to their careers and routine work. This issue is still controversial and require further research.