2.116 At the same time as recognising the international quality of the top end of the LSET
‘product’, some concerns were raised about the consistency of levels of attainment across both academic and professional awards. These concerns were particularly apparent in the Bar and solicitors’ professions, where there is a greater diversity of providers and ‘product’.
It is possible that at least some of the variations in views recorded earlier in this chapter are reflective of that diversity and the variety of experiences that the system accommodates.
The issue of consistency cannot be addressed in the context of most forms of work-based learning across the sector because, currently, any appraisal is a matter primarily for the employer/supervisor, without significant external comparison or validation.
51 Taking the two most established global rankings, in 2011/12 the UK had 10 in the THE world ranking top 100 (in order of ranking): Oxford; Cambridge; Imperial College. London; UCL; Edinburgh; LSE; Manchester; Kings College, London; Bristol, and Durham, with a further 21 institutions in the top 200. By contrast 19 UK institutions made the equivalent QS top 100: Cambridge; Oxford; Imperial College. London; UCL; Edinburgh; Kings College, London; Manchester; Bristol; Warwick;
Glasgow; LSE; Birmingham; Sheffield; Nottingham; Southampton; Leeds; Durham; York, and St Andrews.
52 In a letter to the LETR research team from a ‘Magic Circle’ firm, backed up by comments from the College of Law (now the University of Law).
53 Ibid.
2.117 Two particular points of inconsistency were mentioned in the LETR research. Certain respondents identified inconsistencies in the breadth or depth of coverage of the ‘core’
subjects provided under the prescribed curricula of different universities. Others complained of inconsistency of standards and comparability as between equivalent qualifications awarded by different institutions. Responses raising inconsistency as an issue included some key institutional respondents and providers.
2.118 In relation to the law degree, respondents reflected more widespread concerns about ‘grade inflation’, the increasing numbers of upper seconds, and perceived variability between them:
I think the focus on the standard of degree is perhaps a little outdated now. I think degrees vary so widely and what a 2:1 means at one university is vastly different from another. So I’m slightly dubious about that as a standard.
Barrister
The problem is the diversity. Law degrees vary in quality and the assessment methods used.
Academic (online survey)
This was seen to create selection difficulties for employers, particularly as recruitment decisions are increasingly being made on (predicted) degree grades, rather than
performance on the LPC or BPTC. The point was also made that uncertainty over standards could have an impact on diversity, as recruiters were wary of moving away from tried and trusted ‘brands’. Interestingly, given the degree of discussion about consistency on the QLD, concerns about consistency barely arose in discussions about the GDL which covers the same ground.
2.119 The LPC generated a number of strong views, where concerns about consistency uncovered larger issues for some about the programme’s perceived lack of underlying rigour. As the comments below indicate, variation in assessment methods and practices may account for much of the perception of inconsistency.
LPC courses are more or less farcical when one institution has, for example, a take away drafting assessment and other institutions do not. Students are quite able to get others to do their work for them when it is not done in supervised conditions. One student complained on failing the LPC that he would have been able to do the drafting assessment in non-examination conditions at another institution and would thereby have passed. The process of external examining is a very basic safeguard, one step up from mere face-saving, based on subjective views of a range of individuals with varying experience and little authority to deal with their concerns. They maintain a certain rigour but the resultant assessment instruments allow competency to be measured on a wide spectrum.
Academic (online survey)
The vocational stage of education for solicitors is currently characterised by varying standards of provision across those institutions that offer the programme. While the providers are currently ‘policed’ by SRA external examiners, LPC assessments are nevertheless of variable standard and content. This can be confusing for employers and students.
Kaplan Law School response to Discussion Paper 02/2012
Current assessments are not assessed vigorously enough and are unrealistic, with little practical application.
Junior Lawyers’ Division response to Discussion Paper 02/2012
rent Sy stem of Le g al Ser vices Educa tion and T raining
2.120 CILEx courses and the BPTC take contrasting approaches to assessment from the LLB and LPC. CILEx level 6 law and practice papers are set centrally, ‘so teacher and examiner are separate: students can only rely on past papers to anticipate what might be in an exam. We believe this creates a consistent and reliable standard’.54 It is notable that, in interviews and focus groups, no issues of consistency were raised, though there was some recognition that consistency was bought at some cost to both flexibility and, in assessment, verisimilitude to practice, though the latter problem is not unique to CILEx course assessments.
2.121 The BPTC, by comparison, has moved to a half-way house with a mix of internal and centrally-set assessments.55 This change followed the 2008 Wood Report’s
recommendation that the knowledge-based areas (civil and criminal litigation, professional ethics), should be assessed by a mix of centrally set and assessed multiple choice questions, and short answer questions that are centrally set but marked locally against a standard set of assessment criteria.
2.122 The data gathered does tend to suggest that any move to more centralised assessments is likely to be controversial. For example, while it is supported by Kaplan, based on their experience of QLTS, it is regarded by the University of Law as a retrograde step which, by separating the assessment from the learning, would take the LPC backwards (see responses to Discussion Paper 02/2012).56
2.123 Implementation of centralised assessments can be challenging, as demonstrated by the change process undertaken by the BSB, (acknowledged in response to Discussion Paper 01/2012). This was exacerbated by student complaints regarding the fairness of the summer assessment in 2012. A small number of comments was received on this from students, providers and practitioners. One of the more constructive highlighted the importance of building in sufficient lead-time and trialling into any change process, and possible implications for the LETR.
[O]ur experience of introducing central assessments, which has come from the BPTC, has not been smooth and so if there was to be any sort of transition in the future to more, I don’t know, national qualifications which I suppose is a possibility I would think there really has to be quite a long lead-in time to that - lots of trialling and very strong systems in place before that was introduced ... I think there’s been a lot of things that it’s thrown up that we perhaps hadn’t expected and it’s been a tremendous amount of work. I think if you spoke to anybody from any of the providers I think they’d say the same thing.
Academic (online survey)
2.124 The question then is, if inconsistency is an acknowledged weakness of the current system, whether there are mechanisms - such as those employed in the QLTS - that can more robustly secure a consistent standard of attainment, and the extent to which standing-setting or assessment need be centralised?57
54 CILEx response to Discussion Paper 02/2012.
55 Licensed conveyancer and patent attorney examinations are also set centrally. Other smaller professions have centralised assessment by default where they use a single educational provider.
56 The reasons why the centrally-set Solicitors Final Examination was replaced by the LPC seem to be argued as follows. The curriculum was driven by the requirements of a centrally set examination, thus teachers felt no ownership of it. This led to unimaginative teaching and rote learning. A return to that system could remove the flexibility that enables LPC programmes to be tailored to the needs of individual firms or types of legal practice. A centrally set examination cannot cater for the assessment of practical skills. A major criticism of the former Final Examination, and of the programmes leading to it, was that trainees entered employment with wholly inadequate skills. To the extent that the Final Examination addressed skills at all, it was by expecting students to write about them, rather than to demonstrate their possession. There would therefore be, it is argued, significant risks to the quality and relevance of the LPC.
57 See preceding footnote.