Most of the surveys include items assessing aspects on which students have no information or sufficient experience to provide a reliable assessment.
These must only be answered when students have suitable knowledge of the aspect to be assessed, as in the University of León, or introducing the answer option ‘DK’ in all items7. Judgement based on conjecture and not known facts is to be avoided. For example, in most surveys students are expected to provide opinions on the suitability of criteria and evaluation systems for teachers before completing the subject. They have to indicate whether these comply with the student prospectus, adapt to the subject matter and whether they have been of use for learning and assessment.
We recommend answering the surveys after the assessment process is completed but before the publication of the final marks for the subject, so that these cannot subjectively influence the assessment of the teacher/
examiner. Deliberate revenge for low marks or a subtler process such as cognitive dissonance could lead to low ratings (Boysen, 2008). It is possible to carry out the surveys after the final assessment of the subject in only 13 (27.6%) universities, although they are usually handed out earlier in order to prevent a predictable drop in participation. In order to guarantee higher participation, students could be encouraged with a participation certificate or something similar. In some universities, such as Simon Fraser in Canada, draws are held for the students who have completed assessment questionnaires8 (Simon Fraser University, 2017).
Although the bias of the students not completing the survey is unknown (Wolbring, 2012) they tend to be low-performing and dissatisfied students (Dobkin, Gill y Marion 2007).
Various studies conclude that the online surveys have significantly lower ratings than those carried out in class (Nowell, Gale, y Handley, 2010). Therefore, if students are able to complete these online they should
(7) Only 51% of surveys and 54.4% of items offer this option.
(8) In the 2016-2017 academic year the prizes were an iPad, and gift vouchers to the value of 100$
and 50$.
be given the chance to complete them in class (Treischl y Wolbring, 2017).
We have observed that the assessments are aimed exclusively at compulsory, basic or elective subjects. However, this does not apply to work tutored by the teacher in external placements or the end-of- degree project, which require major organisation, preparation, tutoring, attention, guidance and review. This omission results in these teaching activities being undervalued, as good or bad practice of the teacher in this respect would have no repercussion and teachers would be deprived of information that would be valuable for improving their work.
As stated by González et al. 2010 some of the items are irrelevant or poorly thought out. They are often unsuitable or the wording is poorly formulated, including more than one variable for evaluation. This deprives the teachers of accurate information which would allow them to learn more about their teaching and improve it. In fact, drawing up a tool made up of well-formulated and presented items is crucial to ascertaining the level of satisfaction among students regarding the teachers’ work, with results impacting their professional careers (Moreno-Murcia et al. 2015).
In our opinion, both teachers and students should be surveyed on the aspects to be evaluated in these questionnaires. The relevance and suitability of items should also be verified before being included in the final version, especially bearing in mind the lack of consensus among teachers and students on what constitutes high-quality education (Nasser- Abu, 2017) and an efficient teaching-learning process (Bosshardt y Watts, 2001). There is only one instance, in the University of La Rioja, where students are asked about their knowledge of the teachers’ assessment system and whether they consider the survey items suitable.
The most widespread categories in the surveys, and those considered most important, are the usefulness of didactic resources, the atmosphere of participation promoted by the teacher and the clarity of explanations.
Therefore, other aspects such as the encouragement of reflection and critical thinking, usefulness and accessibility of contact hours, open questions to compile opinions on positive aspects, and suggestions for improvement of teaching activity are in the minority and considered less important.
The differences between some subjects and others which are more prone to the use of didactic media and support considered important a priori could lead to disagreement among teachers. This would partly be
the result of different interpretations of freedom of teaching, as laid down by the Spanish Constitution (art. 20.1.c.) and the Charter of Fundamental Rights of the European Union (art. 13).
It is striking that from a total of 711, only 29 (4%) open items are counted. We agree with Van Gelder’s (2017) assessment that standard questions are very general, require a very simple rating from students and do not reveal a precise opinion. The inclusion of open questions is advised so that students may express their opinions freely, thus enabling more accurate and useful data to be collected.
We propose the creation of a homogeneous teacher evaluation system adapted to the European Higher Education Area (Sabri, 2013), respecting the autonomy of individual universities and including a category to examine the particular characteristics of each university (Mittal et al.
2015). In 2007 ANECA implemented the DOCENTIA program to support the assessment of university teachers’ activity, although it is not yet used in all Spanish universities (ANECA, 2015). This initiative contemplates each university controlling the timeframe of the student satisfaction surveys and the evidence compiled from them. This would cover items relating to the self-assessment of the teacher, curricular teaching merits, reports from academic supervisors and result ratings from the subjects taught.
Although the assessments provided by the students are a means to improving the education they receive (Rantanen, 2013), we do not agree with teachers being categorised and being made responsible for their respective institutions (Baez, 2014). Recent meta-analysis of multisector studies did not report significant correlations between student assessments in these surveys and their learning (Spooren y Christiaens, 2017).
These surveys may reflect a biased view of reality (La Serna, Becerra, Beltrán y Hongrui, 2014), as the student is not necessarily aware of the contextual variables affecting teaching, such as the resources available to teachers in class (Díaz, 2015). There is also an erroneous starting premise that the anonymity of students, regardless of whether they attend class or not, guarantees a sincere response and an objective and accurate assessment of the teacher’s individual teaching skills. This point is highly debatable, given the wealth of aspects which can significantly affect student satisfaction with teachers, including the perception and expectations of individual students regarding teaching-learning, connection with the teacher (Feistauer y Richter, 2017), and the charisma or personality of the teacher providing a clear ‘halo’ effect (Mittal et al. 2015).
Furthermore, there is no consensus on which competences ensure that teachers provide a high-quality education. Although there is a belief that good teachers require their students to make an effort, students do not always agree, and this may result in negative assessments (Braga, Paccagnella y Pellizzari, 2014). In some universities in the Netherlands this has led teachers to question the advisability of reprimanding or being strict with their students, given the risk of being given lower scores in their assessment (Van Gelder, 2017). Some studies also state that the standard satisfaction questionnaires used are not suitable for evaluation of subjects or teachers whose objectives and methodology differ significantly from the conventional approach (Lee Hansen, 2014).