EL PELIGRO PROCESAL COMO PRESUPUESTO MATERIAL DE LA PRISiÓN PREVENTIVA
1. CONSIDERACIONES GENERALES
5.4.1 Data Collection Methods 112 - 116
5.4.2 Data Analysis Methods 116 - 120
- 94 -
5. Methodology and Methods
5.1 Chapter Overview
“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories [...].”
Doyle (1891, p. 3)
As an inexperienced researcher carrying out independent research for the first time, I believe this chapter is one of the most important in the whole thesis. It is therefore not a coincidence that it sits in the middle of my thesis, bringing together the scope of my research, my theorisation of Essential Skills integration into the Welsh Baccalaureate curriculum, and the accomplishment of the empirical parts of the study. My aim in this chapter is to present how the study unfolded and the practical and methodological restrictions and opportunities that shaped its course. In fulfilling this aim, I justify and evaluate my methodological choices, I give detailed descriptions of the six education sites that I accessed and a full account of the data collection and analysis approaches and methods that I followed. In this chapter, I also make apparent how practicalities such as access to participants and data and analytical considerations prompted me to carry out the study in two distinct but interrelated parts. Both of these parts contribute towards meeting my wider research aim of examining the extent to which the Welsh Baccalaureate brings together academic (general) and vocational education and provides a common learning Core to all students irrespective of their Optional choices and site of learning. Figure 4, which spreads over the next two pages, is a graphic representation of the course of the study and it summarises the descriptive content of this chapter.
- 95 - Figure 4. The course of the study.
Personal Motivation: Intrigued by the (policy) promise of the Welsh Baccalaureate Advanced Diploma to bring together academic (general) and
vocational education and to provide a common learning Core to all students irrespective of their Optional choices
and site of learning
(STUDY YEAR 1 - 10/2012 - 10/2013) Part ONE Objectives:
(1) Establish the proportion of students that combine academic (general) and vocational
Optional qualifications (2) Identify students' choices of Optional programmes of
study/qualifications
Part ONE Data: Reported data on WBA awards (WJEC, 2008a; 2008b;
2009a; 2009b) - Triangulate with reported data on A level examination
entries for schools (WG, 2011a; 2012a; 2013b; 2014)
Part TWO Objective: Comparatively describe the integration of Communication and Application of Number Essential Skills with different
components of the Welsh Baccalaureate upper-secondary curriculum Part TWO Methodology: Twin (Communication and Application of Number) Comparative Case Study (STUDY YEAR 2 - 10/2013 - 10/2014 - WBQ Core grading implemented) Sites Accessed and
Characteristics:
Suburban School
(Experienced in WBQ, High Student Attainment, Low
Deprivation Score)
Urban School I (Experienced
in WBQ, Low Student Attainment, High Deprivation
Score)
Part TWO Data: Two semi-structured interviews: WBQ Co-ordinators
(One from each school) Data Analysis Approach:
Hybrid thematic approach (Fereday and Muir-Cochrane,
2006) Part ONE Additional Objective: Establish the purpose and practicability of integrating Communication and Application of Number in the WBQ Part ONE Methodology: Multi- methodological approach (Mingers and Brocklesby, 1997) Observable social trends, policy implications and actors' views
Part ONE Additional Data: Documents: Two policy documents (WAG, 2002c; 2004).Teaching specifications (WJEC,
2013a) and Essential Skills Wales standards (WAG, 2010). Interviews: 1 question from a total of 14 semi-structured interviews carried out for Part 2 (Years 2 & 3) Data Analysis Approaches: Textual (corpus-based) exploratory analysis and framework analysis (Richie and Spencer,
- 96 - Figure 4 (cont.)
Sites Accessed and Characteristics: Faith School (Inexperienced in WBQ, High Student Attainment, Low Deprivation Score) Urban School II (Very experienced in WBQ, Low Student Attainment, High Deprivation Score)
Part TWO Data: Four semi-structured interviews: 2 WBQ Co- ordinators (Faith School and Urban School II), 1 Literacy ESW Co-ordinator (Faith
School), 1 WJEC
Data Analysis Approach: Hybrid thematic approach
(Fereday and Muir- Cochrane, 2006)
Part TWO Data: Documents: 4 Portfolios of Student Work, 6 Assessment Specifications (awarding bodies), 4 Principal Moderators' Reports (awarding bodies), 2 Chief Investigators' Reports and 2 Examiners' Reports (awarding bodies), 3 Internal WBQ Reports (WJEC) Data Analysis Approach: Directed qualitative content analysis approach (Hsieh and Shannon,
2005)
(STUDY YEAR 3 - 10/2014 - 10/2015 - the New WBQ
implemented) Sites Accessed and
Characteristics: Urban Campus of FE College (Very experienced in WBQ, High Student Attainment in WBQ, High
Deprivation Score) Exurban Campus of FE College
(Very experienced in WBQ, High Student Attainment in WBQ, Very
High Student Attainment in Key Skill Qualifications, High
Deprivation Score)
Part TWO Data:
Six semi-structured interviews: 2 WBQ Co-ordinators, 2 Literacy/ESW Co-
ordinators, 2 Numeracy/ESW Co- ordinators (One from each college) Data
Analysis Approach: Hybrid thematic approach (Fereday and Muir-Cochrane,
2006)
Documents: 2 curriculum model plans (Urban college) Data Analysis Approach:
Directed qualitative content analysis approach (Hsieh and Shannon, 2005)
Part TWO Data: Two semi-structured interviews: WBQ Co- ordinator Suburban School,
WJEC Official Data Analysis Approach:
Hybrid thematic approach (Fereday and Muir-
Cochrane, 2006)
Part TWO Overall Analytical Strategy: Two-tiered multiple case study (Stake, 2006)
comparisons
1. Each case study (Communication and Application of Number) as a distinct ideographic event (within each of the 6 educational sites and
cross-site comparisons)
2. Cross-case study (Communication and Application of Number) comparisons
- 97 -
5.2 Part 1 of the Study: Objectives and Methodology
Contrary to the experience of many of my fellow PhD students, my research did not naturally grow out of previous studies in sociology, independent reading or upper-secondary teaching experience in Wales. Seidman (2006, p. 43) notes that in such cases, inexperienced researchers can see the work involved as a requirement and a hurdle to overcome, denying themselves the pleasures and opportunities that come from doing research. This certainly held true for me in the first part of the study during which I had to establish a focus for my research, to review a wide range of literature and to gain an understanding of the upper-secondary education system in Wales.
My original intention in this part of the study was to describe who the Welsh Baccalaureate students are, what their Optional choices are and how these choices have been changing since the introduction of the qualification. I envisaged classifying students according to socio-demographic (age, sex, neighbourhood economic deprivation status, language of instruction) and educational characteristics (type of institution: college/school; previous study and attainment) with the possibility of examining the likelihood of different students following different Optional qualifications. This intention never materialised and despite early agreements with the WJEC, I was not able to get access to raw data from the organisation. Although the WJEC co-funded the study, and one of my points of contact there was particularly helpful to me, I never managed to establish a fruitful relationship with key gatekeepers. To this date, I am uncertain whether limited access to raw data was due to a lack of the type of data I required for the study or an unwillingness of the organisation to share it with me.
The three research objectives of this part of the study were to:
1. Establish the proportion of students that combine academic (general) and vocational qualifications as part of their Optional studies
- 98 - 2. Examine how Essential and Wider Skills are portrayed in official pedagogic documents of the Welsh 14–19 education context
3. Examine how key agents involved in curriculum development and practice perceive the purpose and practicability of integrating Communication and Application of Number in the Welsh Baccalaureate.
I focused exclusively on these two Essential Skills, motivated by historical, theoretical and practical considerations. First of all, these are the two Skills that can be considered part of the ‘staple’ provision and proposals since the FEU (1979) A Basis for Choice report, through to the Curriculum 2000 Key Skills (QCA, 1999) and to the establishment of Essential Skills (WAG, 2010). Secondly, and with curricular intentions for Essential Skills to be both “ends in themselves for setting up and working out problems and as transferable skills called upon in more general contexts” (WJEC, 2013b, p. 5), I decided to choose two distinct ‘species’ of genericism that resemble disciplinary subjects (i.e. English/English Literature and Mathematics). Finally, in order to follow a multi-layered analytical strategy, I decided to limit data to two case studies and avoid making the study unnecessarily complex to carry out and report.
In meeting the diverse research objectives of part 1, and by drawing mainly from the tradition of empiricism, I followed a multi-methodological approach (Mingers and Brocklesby, 1997) that combined an examination of observable social trends, policy implications and agents’ perceptions. This multi- methodological approach allowed me to capture many contextual aspects of the integration of Essential Skills into the upper-secondary curriculum as well as to examine the extent to which the Welsh Baccalaureate is bringing together academic (general) and vocational qualifications under its overarching Diploma.
- 99 - 5.2.1 Data Collection Methods
In establishing students’ choice of Optional qualifications, I collected data that was already reported by the WJEC (WJEC, 2008a; 2008b; 2009a; 2009b). With the data suggesting that students may be switching from academic (general) to vocational qualifications, I triangulated this finding with statistics reported by the Welsh government (WG, 2011; 2013b; 2014) on A level examination entries for schools. Overall, it has been difficult to establish any clear patterns and make interpretations as there were discrepancies with the data reported by the WJEC. Furthermore, this data only covered the third (2005–2007), fourth (2006– 2008) and fifth (2007–2009) examination cycles. Nonetheless, I draw some useful conclusions from this part of the analysis in relation to the role of qualifications, the repositioning of academic (general) and vocational qualifications, and the resilience of traditional A level subjects.
In examining how Essential and Wider Key Skills are portrayed in official pedagogic documents relevant to the upper-secondary curriculum, I obtained the WJEC Teaching Specification document for the Welsh Baccalaureate Advanced Diploma (WJEC, 2013a) and the Essential Skills Wales standards document (WAG, 2010) published by the Welsh government’s Department for Children, Education, Lifelong Learning and Skills (DCELLS). I also collected two key policy documents, Learning country: Learning pathways 14–19 consultation paper (WAG, 2002b) and Learning pathways 14–19: Guidance (WAG, 2004), that I used for an exploratory, inductive, corpus-based analysis.
Finally, in gaining an understanding of what the purpose of Communication and Application of Number in the upper-secondary curriculum is, I carried out 14 semi-structured interviews (table 8) with a WJEC official, WBQ Co-ordinators and Literacy/Numeracy/ESW Co-ordinators at different schools and colleges. I initially carried out the first two interviews at the very early stages of the second part of the study. As I was still experimenting with my interviewing techniques and questions, I found that the data I collected through these two interviews was more suitable to be analysed as a complementary segment of the first part of the study. I then went on to include the following question in my remaining 12 interviews: “What is the purpose of Communication and
- 100 - Application of Number in the upper-secondary curriculum of the Welsh Baccalaureate?”
My approach to interviewing can be described as top-down (Pawson, 1996; Smith and Elger, 2012, p. 12), meaning that although I recognised the expertise of interviewees and the fact they had privileged access to practices, motives and understandings, I played an influential role in framing their thinking and the accounts they offered me through my semi-structured interview schedule and theoretically informed conceptions of the research topic. I discuss my interviewing approach in more detail in sub-section 5.4.1 of this chapter.
Table 8. Interviews carried out
# Participant’s Role Education Site Duration
(minutes)
1 WJEC official WJEC ~40’
2 WBQ co-ordinator Suburban school ~52’
3 WBQ co-ordinator Suburban school ~48’
4 WBQ co-ordinator Faith school ~70’
5 WBQ co-ordinator Urban school I ~83’
6 WBQ co-ordinator Urban school II ~85’
7 WBQ co-ordinator Urban campus of college I ~71’ 8 WBQ co-ordinator Exurban campus of college
II
~90’
9 Literacy co-ordinator Faith school ~85’
10 ESW co-ordinator Urban campus of college I ~72’ 11 ESW co-ordinator Exurban campus of college
II
~84’
12 AoN/Numeracy co-ordinator
Urban campus of college I ~65’
13 AoN/Numeracy co-ordinator
Exurban campus of college II
~75’
- 101 - 5.2.2 Data Analysis Methods
After composing a corpus of 30,324 words based on the two policy documents and a corpus of 10,537 based on the teaching specifications, I ran a concordance (using AntConc version 3.3.5w) to see which were the most frequently used words in these documents (excluding articles, prepositions, connectors and auxiliary verbs) and their collocations (i.e. their tendency to co-occur with other lexical items in the same sentence). This analytical work was in line with Stubbs (2001, p. 24) and I also examined the main collocations of a list of key words for the study (i.e. academic, vocational, qualification*, knowledge, curriculum). This type of analysis helped me see which words were over-represented or under-represented in the text, and by using the KWIC (Key Word In Context) feature of the concordance, I was able to see particular key words (‘nodes’) in word sequences (see figure 5). My objective during this basic textual analysis was to explore the relations between different curriculum elements and the development of interpretative frames about these elements as ‘universal provisions’ for education. The basic assumption guiding this type of analysis was that “the meaning of a word is fully reflected in its contextual relations” (Cruse, 1986, p. 16) and that once I established semantic relatedness and meaning in one context (e.g. the policy document), the meaning can be abstracted and compared to a different one (e.g. the teaching specification) and eventually in the accounts the interviewees offered me.
Figure 5. KWIC view of ‘qualification’ in Learning Pathways 14–19: Guidance
- 102 - I then analysed the teaching specification document, the Essential Skills guidelines and the interview data using framework analysis (Richie and Spencer, 1994; 2002). This approach was originally designed for carrying out applied policy research and I found it suitable for dealing with contextual and diagnostic research issues (Richie and Spencer, 2002, p. 307) (see table 9). Research in this part of the study was therefore driven by the need to collect specific information rather than theoretical concerns.
Table 9. Contextual and diagnostic research interests (Adapted from Richie and
Spencer, 2002, p. 307)
Category Objectives Research Question Contextual Identify the form and
nature of what exists What elements operate within a system? What are the dimensions of attitudes or perceptions that are held?
Diagnostic Examining the reasons for,
or causes of, what exists What particular factors attitudes or underlie perceptions?
Why are actions taken or not taken?
Overall, framework analysis allowed me to maintain a systematic approach to the analysis and interpretation of different types of data (documents and interviews) while still retaining the flexibility to identify data-driven issues that helped me sharpen the focus of the second part of the study. It also allowed me to complete the analysis of the interview data for this part of the study at different stages of the research project and to eventually interpret findings within a single conceptual framework. In my framework analysis, I followed five steps:
1. Familiarisation with the data – transcribing. By reading and re-reading the
data, I was able to gain an overview of what participants were ‘telling’ me in relation to the predetermined questions. In a sense, this was my entry point to the research. Considering that my involvement as a researcher/interviewer can
- 103 - be encapsulated in Pawson’s notion of teaching interviewees “the overall conceptual structure of the investigation to the subject” (1996, p. 305), this was an important stage of the analysis as my first opportunity to read interviewees’ accounts. For the documents, this stage involved reading the whole document, annotating it and re-reading parts that I found of relevance based on the initial lexical analysis.
2. Identifying a thematic framework. This stage involved identifying and
amalgamating issues that emerged from the data (e.g. how Essential Skills are used to ‘boost’ grades to apply to specific university courses) and a priori issues I had identified through my engagement with the literature (e.g. from the distinction between different types of knowledge, and from the problem of early specialisation in post-16 education). From this process, I created ‘labels’ that I then assigned to sections of the data.
3. Indexing. Stages 2 and 3 were interconnected through the process in which I
assigned labels to small sections of the data (sentences or short paragraphs) with the assistance of NVivo software. For the documents, this stage involved annotating passages using different coloured markers. One of the challenges while indexing interview data was interpreting participants’ intended meaning, especially if I found the wording unusual. For example, in the following passage, the WBQ co-ordinator of a college explains how tutors are not keen on teaching Essential Skills. This is linked to teachers’ view of Skills as remediation for past education ‘failings’ and to time constraints. ESW co-ordinators can help tutors ‘embed’ (integrate) Skills in their vocational programmes.
“I think it [Essential Skills] puts an added pressure on subjects, because it’s obviously what we have to do now. Not only have you got to teach your subject, you have now got to teach Literacy and Numeracy on top […] Maybe if you put our groundwork into Literacy at the beginning, it does make them better students for your subject later on. People who put the
graft in at the beginning to really try and promote Literacy and
- 104 - In this passage, I did not know what the interviewee meant by “groundwork” and “put the graft in”. Lacking first-hand contextual knowledge, I often relied on knowledge I obtained through my engagement with literature and through interviewing other participants, as well as contextual clues from other parts of the interview. For example, I initially interpreted “groundwork” and “put the graft in” as linked to a type of instruction and did not understand what was meant by “better students”. Later in the interview/transcript, the interviewee made clear what “putting the graft in” was linked to: allocating time and staff to help with ‘signposting’ opportunities and materials to teach Communication:
“It is a matter of allocating time and resources to it. Although it is a pain, perhaps it is a necessity. The commitment to teaching the basics needs to take priority and people who do promote it, they do create better students when it comes to being assessed
and put[ting] together the portfolio [...]. Once they understand that they are not alone in the task and that there is help in aiding them [with] how to present a lesson plan, schemes of work, understand how to
embed it, they will work on it.”
In the second part of the study and after having interviewed a number of WBQ co-ordinators who acknowledged the assessment-led approach to integrating Essential Skills, it became clearer that the interviewee was also referring to a further issue: the dual roles of tutors as instructors and assessors and the privileging of the ‘basics’ of capitalisation, punctuation, paragraphing and spelling. I believe that my discussion of the dual role of tutors in chapter 8 makes my indexing example more meaningful.
4. Charting. During this stage, I reduced, summarised and ordered all data by
assigning them to the ‘labels’ of the thematic framework I developed in steps 2 and 3. It was during this process that I identified the categories/themes to which each chunk of text related. (See appendix 5 for ‘charting’ the
instrumentality theme.) For the documents, I assigned handwritten and colour-
- 105 -
5. Interpretation. This was the final stage of synthesising all themes and
identifying clear links with issues I had already identified in the literature. This interpretation is part of the argument I build in chapter 6.
Overall, the first part of the study was a necessary stage of the research process that helped me gain an understanding of upper-secondary education in Wales, develop my research interests, sharpen the focus of the study and provide a contextualising basis for interpreting the findings of the second part, a part that