La población joven estudiante y titulación alcanzada

Lincoln and Guba (1985: 276) wrote that ‘Documents and records are singularly useful sources of information, although they have often been ignored, particularly in basic research and in evaluation.’ Despite the importance of documentary evidence for contemporary societies, ‘British and American social scientists have never been entirely confident about analyising written texts (Silverman, 2004: 153) but rather the methods are focused on the analysis of speech and action. Silverman argued that ‘the mere act of transcription of an interview turns it into a written text’. Documents are often seen as inappropriate or secondary sources, unable to indicate sufficient insight into systems of social meaning and practice. Further, many researchers are cautious about using documents in education research, given uncertainties about the most appropriate strategies to use and the limited guidance available in much of the qualitative methodological literature (Guba and Lincoln, 1981: 231-232). Yin (2009) wrote that documentary evidence is relevant to every case study. The examination of documentary evidence in the case study of an educational programme is used as a resource to get a comprehensive and objective picture of how the programme operates in practice and how is implemented (Silverman, 2001).

This section describes how the documentary evidence is used in qualitative educational research, as supplementary and contextual source of data (Guba and Lincoln, 1981). This discussion includes the nature of documents, distinctive dimensions of research with documents, the approach to analyse documentary evidence and how this documentary evidence is used to supplement other data collected in this study to develop a comprehensive and coherent understanding of the programme, its setting, processes and outcomes.

DEFINITION OF DOCUMENTS AND RECORDS

The definitions of the terms ‘document’ and ‘records’ often overlap and they are used interchangeably. In the Cambridge Dictionary (Cambridge Dictionaries on-line, accessed 09/2009), ‘document is defined as ‘a paper or set of papers with written or printed information, especially of an official type’; ‘a text that is written and stored on a computer. Similarly ‘record is defined as ‘a piece of information or a description of an event which is written on paper or stored on a computer’; ‘the facts that are known about a person or a company and the actions they have done in the past’.

Guba and Lincoln (1985: 277) defined ‘document’ as ‘any written or recorded material other than a record that was not prepared specifically in response to a request from the inquirer’. They used the term ‘record’ to identify ‘any written or recorded statement prepared by or for an individual or organisation for the purpose of attesting to an event or providing an accounting’.

Apart from the difficulty in defining these terms, Guba and Lincoln (1981: 229) further distinguished the different typologies of documents and records. For the purposes of the this study, only relevant typologies are mentioned here: primary and secondary evidence - all the documentary evidence (this term refers to both documents and records) examined in this study is primary which means that it was generated from the direct experience of ‘a particular situation or event’ rather than from other sources (not someone’s description or summary of them). This is important in checking the authenticity, verification, reliability and objectivity of documentary evidence. Linked to this is another typology that is important to this study and that is solicited versus unsolicited (controlled or imposed by someone), comprehensive versus limited (in the case when documentary evidence is limited for confidentiality reasons e.g. in staff meetings students’ initials are used so this document was only analysed for the meeting agenda topics rather than the contents of the actual minutes), edited versus unedited (controlled for contents) (Bogdan and Taylor, 1975 cited in Guba and Lincoln, 1981: 229).

In choosing the definition of documents and records and identifying the typologies of these terms, it is important to refer back to the objectives set in this phase of research

which is essentially descriptive. The documentary evidence provides the information on the context of the programme, its aims and objectives, its outcomes, its participants, description of their needs and how these needs are addressed by the programme processes, resources and outcomes. This will inform the research question 3 posed by this study: ‘How can Bronfenbrenner’s ecological systems theory guide an examination of an alternative educational programme?’ It will also assess ‘How effective is the NHP in catering for students’ needs and tackling their disengagement?’ which emerged as a sub question.

For the purposes of the current study, the term ‘record’ will be used to describe any written material created by the Local Authority or Senior Management and in which the staff and programme management did not have an input. The term ‘document’ will be used to denote any written material in creation of which the programme staff and management have had their input. I have chosen this classification in order to minimise biases which could potentially arise due to my involvement on the programme. Hence, a researcher in education should be open to different perspectives and viewpoints to ensure comprehensive examination of educational programme to check for quality.

THE ADVANTAGES OF USING DOCUMENTARY EVIDENCE

In the context of an internal researcher, there are many advantages of using documentary evidence. First, it is easily accessible since all the programme documents examined in this study have been open for examination to all the teaching staff members not only the researcher. Second, it is an inexpensive source of evidence demanding mostly researcher’s time, effort and patience. Third, it is appealing to the internal researcher due to its stability as a source of information in terms of accuracy, authenticity and unchangeability over a period of time so that it can be re-examined by other researchers. Fourth, it is an invaluable source of information ‘grounded’ in the context of the programme since some documents have been specifically designed for the programme or adapted to the context of the programme and its participants (such as ‘Student Programme Reviews’, ‘Interview checklist’, Individual Learning Plans’, Student File’) by the programme staff or in consultation with the staff. Finally,

it is an ‘unobtrusive’ source of evidence which was ‘not created as a result of the case study and as such it cannot be altered by the researcher’ and thus ‘contaminated’ by the potential biases (Yin, 2009: 102).

The study of documents provides access to events that cannot be observed, to examples of formal communication about the programme processes, and to social actors that generate meanings and practices. By comparison with qualitative research of live talk and action, in which researchers must actively generate new social data, the pre-existing status of documentary evidence makes them distinctive and results in the need for explicit attention to document selection, and to the intended nature of these textual resources. They are not produced by the researcher or for this study so they are less biased. For this reason, they were reviewed and selected by the programme staff during group discussions.

Bronfenbrenner (1977) criticised the exclusion of scientists from the research process in conducting experiments. Thus, one could argue that the insider researcher is in a more advantageous position because she has access to the programme so she can conduct naturalistic observation. However, Bronfenbrenner (1977:79) argued that this is only possible ‘with the stipulation that it be unguided by any hypotheses formulated in advance and uncontaminated by structured experimental designs imposed prior to data collection’. Thus, in order to avoid imposing any hypotheses that she formulated in the course of her involvement on the programme, in the present study instead of observations, the researcher decided to consult the staff on the most relevant and credible approaches to data collection during group discussion 1. Guba and Lincoln (1981: 232) asserted that documents and records represent a ‘natural’ source of information, they described them as an ‘in context’ source of information which ‘arises from the context [and], exist in it but they consist of information about the context’. As such, they contain ‘well-grounded data’ on the different aspects of the NHP that is being investigated. Qualitative researchers seek to understand the world from a participant’s point of view, by listening to or observing a person in a natural environment. By using documents, a researcher is placed ‘at some distance from real people, so that human action and thought are interpreted through representations of reality’ (Silverman, 2001: 154). However, Yin (2009) suggested that documents can ‘corroborate information from other sources. As such documents can be used as

important resources for data triangulation, to increase the comprehensiveness and validity of any single study (Patton, 2002: 248).

THE PURPOSE OF DOCUMENTS AND RECORDS

Documentary evidence is not produced as an independent personal act but in ways that draw on and relate to other documents. Therefore, the researcher needs to examine why they were made, for what purpose and by whom.

Documentary evidence in this study is a specific type of formal communication and networking amongst the programme staff, management and students within the programme as well as outside of programme with the external agencies interested in the programme, such as parents, referral agencies and the LA. This means that documentary evidence shows the competence, and often the specialized knowledge, of their producers. Their form and content conforms with clear rules (Bauer, Gaskell, & Allum, 2000), such as the rules of collaboration agreement or institutional policies in this study. Documentary evidence is used to record and reflect social arrangements; organisation of social life on the programme, its processes, inputs, outcomes and assess the achievement of aims, objectives and set targets.

Documents and records are not simply containers of meaning. They are actively and collectively produced, exchanged, and used. The production of documents and records involves many decisions, by different people, about what information to write, in what style, for what audience, and for what purpose. They are produced in and reflect specific social and historical circumstances. In this study they are used as supplementary and contextual data to ensure objectivity, validity and reliability of evidence collected through other methods.

The analysis of documentary evidence allows for minimal researcher intrusion. This is an important resource for an internal researcher for whom observation could alter rather than illuminate the social world.

SAMPLING STRATEGIES

The purpose of the current study is to provide an in-depth account of the programme in its context, its implementation, processes, inputs and outcomes. Quantitative researchers sample randomly, to permit generalization, but qualitative researchers sample information-rich cases, to permit in-depth understanding (Patton, 2002). Although generalisations can be made with the aim of providing recommendations for the development and improvement of other educational programmes of a similar context, the primary aim of this study is to provide a detailed investigation of the NHP.

The selection strategy applied in the analysis of documentary evidence in this study is purposeful sampling. This strategy is used to identify attributes other than the individuals themselves for investigation of processes and practices, assessment of programme weaknesses and strengths and implementation of the programme. A purposeful sampling strategy yields information-rich documentary evidence to exemplify actions or situations that are extreme or intense, to demonstrate maximum variation or homogeneity, or to identify typical or critical cases (Patton, 2002). Legal documentary evidence, college policies and programme-specific documentary evidence are unique information about processes on the NHP.

The selection strategy was structured for comprehensiveness. The researcher purposefully selected the most information-rich and appropriate sources in relation to the goals of the research. The selection strategy provides a systematic process and theoretic rationale for choosing among the plethora of available documentary sources. This study draws on Bronfenbrenner’s ecological systems theory and in accordance with this theory aims to capture all documents and exchanges from different agencies in order to assess the relationships between different systems.

Since these textual resources are socially produced beyond the control of researchers, the researcher must attend to selecting available documents, considering what documents are not available, and conducting some degree of ‘source criticism’ of the selected documents (Howell and Prevenier, 2001). Source criticism has both external and internal dimensions. External critiques to establish authenticity and accuracy are technical including consideration of where, when, and by whom a source was created. Internal critiques establish how a source can inform, by considering intentions and abilities of the document’s producers.

THE EXAMINATION AND ANALYSIS OF DOCUMENTARY EVIDENCE

The examination of documentary evidence formed a descriptive phase of the research procedure. The examination of documentary evidence requires analysis and interpretation. Robson defined it literally as ‘breaking up of something complex into smaller parts and explaining the whole in terms of the properties of, and relations between, these parts’ (Robson, 2011: 412).

Many analytic strategies are available for qualitative interpretation of documents. Atkinson et al. (2001) suggested the following two analytic strategies in qualitative research: content analytic strategies which focus on sources as independent containers of fixed evidence about the social world, and context analytic strategies which focus on sources in ways that embed them in the social contexts of their production and use. For analysis of documents, two distinctive approaches were adopted: (a) the analysis of documents for their content (content analytic) and (b) the analysis of documents as commentary (context analytic)

Although content analysis is a type of inductive analysis for researchers who work with live talk and action, these techniques and theories are adapted in this study for the qualitative content analysis of documents (Altheide, 1996). The term content analysis indicates a range of qualitative analytic strategies to “identify core consistencies and meanings” in a volume of qualitative data (Patton, 2002, p. 453). Content analysis is used to elucidate key patterns, themes, and categories, drawing on previous research, theoretical considerations and research questions.

Researchers who use documents as commentary can adopt a variety of analytic strategies, depending on the research questions they pose, and several analytic approaches are used in this study: description; theoretical presuppositions to structure data analysis; careful attention to alternate or contrary examples or explanations; and the use of multiple types of documents or sources of data for triangulation (Yin, 2003). The analysis of documents as commentary requires significant attention to their socially exchanged and produced nature.

THE ANALYTIC PROCEDURE

Following the outline of the research proposal by the researcher and the initially set research questions, it was agreed in group discussion 1 that the first step in the data collection should be the examination and analysis of documentary evidence that is available to all the staff participants involved in this research.

Robson (2002) recommended that the documents and records needed to be repeatedly read for meanings and patterns. The documentary evidence was collected and analysed using a thematic coding approach. This approach involves ‘coding and labelling’ of all or parts of data to identify whether they represented something of potential interest for this research study and how these data related to the ideas and/or theory about these data (Robson, 2002: 467). The content of the entire data was examined and coded for the specific features of the documents and records. These included their headings, strategies, practices, activities, procedures, tactics and meanings – what aims, objectives, targets, norms, rules, and values guide their action. The initial codes were generated. Codes refer to ‘the most basic segment, or element, of the raw data or information that can be assessed in a meaningful way regarding the phenomenon’ (Boyatzis, 1998: 63).

Codes with the similar label were then grouped together as a theme. Boyatzis (1998: 5) defined a theme as ‘a pattern found in the information that at minimum describes and organises the possible observations and at maximum interprets aspects of the phenomenon’. In the first instance, codes and themes occurring in the data were determined inductively from reviewing the data. In further analysis and interpretation of the data, themes were generated deductively from the research questions posed by

this study, previous research on AEPs and theoretical perspective adopted by this study. The themes were then compared and analysed in all the documents and records that were examined (Glaser and Strauss, 1967 cited in Guba, 1984: 339). Through these comparisons of the themes re-occurring in different sources of documentary evidence, the important features employed on the programme were inferred to explore and describe the programme processes and to assess its strengths and weaknesses. These codes and themes were then presented in a table to enable easier identification of the themes (see tables 4.1- 4.4 in Chapter 4, section 4.2).

In addition to this, the documentary evidence used to record programme aims, objectives and targets; students’ progression and the programme processes was examined in terms of the following questions: who designed the documentary evidence, the purpose for which it was made, how was it recorded, who checked it, was it edited, who has access to them, when and how often was it produced. This explains how documentary evidence relates to each other and checks the authenticity of it. Following this, all documentary evidence was divided based on whether it was solicited, comprehensive or edited and whether the programme staff had an input in their creation. The implications of the above were addressed in the discussion on each document and record in Chapter 4.