NIVEL INTERMEDIO B2

Qualitative research is often thought to produce ‘soft, unscientific results’ (Devine (2002, p.204), yet Silverman (2005, p.209) argued it is not ‘a soft option … [and]

demands theoretical sophistication and methodological rigour.’ Indeed, Smith (2007, p.5) stated that qualitative research is ‘more than method … It is about methodology.’ According to Silverman (2005, p.209), such methodological awareness provides an indication of the quality of research through the researcher’s commitment to demonstrate that the procedures used ensured methods ‘were reliable’ and ‘conclusions valid.’ Validity is essentially another word for truth (Smith and Caddick, 2012; Silverman, 2005), or the ‘truthfulness of one’s conclusions’

(Vaughn and Daniel, 2012 cited in Smith and Caddick, 2012, p.69). More specifically to this research, ‘the extent to which an account accurately represents the social phenomena to which it refers’ (Hammersley, 1990, p.57). Reliability on the other hand, is concerned with the consistency of methods and procedures by which data are collected (Lincoln and Guba, 1985; Gratton and Jones, 2010).

However, the issue of research quality and the appropriateness of reliability and validity, in relation to qualitative investigations, is a contested area (Gratton and Jones, 2010; Bryman, 2012). The slipperiness of the concepts is compounded by the lack of consensus, the proliferation of criteria lists, and the alternative views of reliability and validity (see Lincoln and Guba, 1985; Yardley, 2000; Bryman, 2012;

Smith and Caddick, 2012). According to Gratton and Jones (2010, p.98), qualitative research should be ‘trustworthy, authentic, reliable, rigorous, and credible’, which are seen as a set of operationalising criteria to address the reliability, validity, and quality issues/limitations within this study, particularly as Yin (1994, 2003c) highlighted a lack of reliability and construct validity as a common criticism of case study research.

To address reliability and construct validity issues, this study adopted Yin’s (1994, pp.90-99) recommendations to demonstrate consistency and transparency, including: i) utilisation of multiple sources of evidence, aimed at the development of converging lines of inquiry, and thus a process of triangulation; (ii) creation of a case database to organise and document the data collected from the case studies, including interview transcripts, observations on document analysis, and rigorous recording of references used. Within which the setting, context, participant, research

99 strategy and data analysis methods, are described in a sufficiently detailed manner (Moisander and Valtonen, 2006); and (iii) maintenance of a chain of evidence – the principle of which, is to allow an external observer to trace the derivation of any evidence from initial research questions to ultimate case study conclusions. In addition, this research broadly followed Yin’s (2009) iterative model, which implies ‘a continuous moving back and forth between diverse stages of the research project’

(Verschuren, 2003 cited in Easton, 2010, p.119). In this regard, the analysis of the first case study provided an initial insight and further understanding of the concepts used in the analysis of the data, and together with the analysis of subsequent cases made certain that a reflective and reflexive approach was adopted for the whole research process. The use of three case studies serving to strengthen validity by providing cross-case analyses for analytical generalisations to theoretical propositions (Yin, 2003a).

Furthermore, authenticity in terms of honesty, accuracy (Mason, 2002), and fairness (Gratton and Jones, 2010), can be seen by the careful selection of participants, based on their relevance to the research questions, the theoretical position, and the explanation or account developed (Mason, 1996) within this study. The same can be said for the confidential and structured nature of the interviews, undertaken by informed consent at neutral times to minimise ‘subject error’, with anonymity stressed to avoid ‘subject bias’ (Gratton and Jones, 2010, p.93). The recording and transcription of interviews allowed for repeated examinations of the interviewees’

answers and public scrutiny of the data to counter accusations of bias, as well as offering alternative explanations to support the judgement of evidence and its interpretation (Lieblich et al., 1998), from a range of different perspectives. Thus, providing a form of ‘within-interview triangulation’ (Dean and Whyte, 1978 cited in Gratton and Jones, 2010, p.167). Reliability, credibility and rigour are demonstrated further through professional scrutiny from supervisors and internal reviewers, and the chosen research strategy, inclusive of the techniques used for data collection and analysis, which were deemed the most logical and appropriate for this research (Mason, 2002). The latter signposting the construct of a coherent and meaningful picture, both internally (how the constituent parts logically fit together – Grix, 2010), and externally, using existing theories and previous research (Lieblich et al., 1998).

100 As alluded to above, a significant, though contested technique for demonstrating validity is ‘triangulation of method’ (Mason, 2002, p.190). Whereby, multiple methods of data collection are utilised to investigate the same phenomena (Mason, 2002;

Gratton and Jones, 2010; Grix, 2010), the aim of which is to share strengths, eliminate weaknesses and corroborate results (Houlihan and Green, 2006).

Arguably, the popular conception that triangulation simply means observing ‘an object of study from different angles using different methods’ is problematic, due to the ‘different ontological and epistemological underpinnings of research strategies, consisting of combinations of methods’ (Grix, 2010, pp.135-136; Mason, 2002). That said, Grix (2010) argued that methods should be viewed as tools for collecting data and should not be looked upon as being automatically ‘rooted in epistemological and ontological commitments’ (Bryman, 2001, p.445). Grix (2010, p.135) elaborated further on this point by stating:

‘As long as you are aware how you are employing specific methods and what this method is pointing towards, and how this relates to the ways in which you employ other methods, there should be no problem. The key point is to check whether your methods are ontologically consistent with one another, and, as a consequence, whether they are epistemologically consistent.’

In addition, it is argued that the use of data triangulation to cross-check data collected from semi-structured interviews and document analysis, ‘attempt[s] to get a

“true” fix on a situation by combining different ways of looking at [the phenomena] or different findings’ (Silverman, 2005, p.212). According to Mason (1998, pp.25-26):

‘seeking to corroborate one source and method with another, or enhance validity and reliability through some form of triangulation of method … you will need to think about on what basis one set of data, or method, can corroborate another. This will involve asking whether the two sets of data tell you about the same phenomena or whether the two methods yield comparable data.’

The data collection methods for this study were considered both complementary and comparative, and seen as both reliable and valid by many researchers (Gratton and Jones, 2010).

101 While data triangulation may give the impression that validity has been strengthened, there is emphasis on the researcher to err on the side of caution, and to remain aware of any potential problems (Mason, 2002; Gratton and Jones, 2010), particularly as the blurred boundaries of reliability and validity can be challenging for the qualitative researcher. However, rather than allow this research to be ground down by such methodological issues (this is not a methodological thesis), it is an awareness of the operationalisation of the concepts of reliability and validity, and the degrees to which they are attainable and demonstrable that has been of importance to this study. The aim being to ensure that the end product of this thesis is a methodologically informed, critical, engaging, interesting and exciting piece of sport qualitative research, which can be held to be ‘valid or true [as] it represents accurately those features of the phenomena that it is intended to describe, explain or theorise’ (Yin, 1994, p.92). As Hammersley (1992, p.69) argued, ‘we must judge the validity of claims [about truth] on the basis of the adequacy of the evidence offered in support of them.’ Thus, findings or conclusions are ‘likely to be much more convincing and accurate if … based on several different sources of information.’

Although for Devine (2002, p.206), ‘all empirical material, be it quantitative or qualitative in kind, is subject to different interpretations’, and so in essence, ‘there is no definitive interpretation that is the “truth.”’

4.9) Periodisation

An integral part of this study has been an understanding of the historical context of contemporary sport policy within which NGBs operate, an analysis of the impact of changes in sport policy on SMNGBs, and an evaluation of the explanatory value and utility of the ACF within the UK sport policy process. In order to fully comprehend the development of UK sport policy, a periodised timeline was utilised to map the evolution of the policy area. While an in-depth debate on the concept of periodisation is beyond the scope of this research, it is acknowledged that periodisation is a contentious matter (see Struna, 1985; Katznelson, 1997; Hobsbawm, 2000;

Lieberman, 2001; Phillips, 2002; Hérubel, 2008; Houlihan and Lindsay, 2013).

102 According to Houlihan and Lindsey (2013, p.5), the dimensions of change within sport policy ‘involves the search for, and analysis of, pattern, trends, key events, continuities and breaks with the past with the aim of giving meaning to a collection of events and policy decisions in a particular time period’, where a metaphor style of periodisation has the potential to highlight surface or superficial change. For example, changes in the way Lottery funding is distributed to sport and how this impacts on the administration of NGBs, or changes that occur at a much deeper, societal level, such as actions by the state or changes in societal values regarding the importance or role of sport (Houlihan et al., 2011). Furthermore, Houlihan and Lindsey (2013) highlighted the usefulness of the metaphor of levels within the ACF to analyse sport policy, while Sabatier (2007) claimed that a time period of at least 10 years is a pre-requisite for an accurate analysis of policy change. Arguably though, a period greater than 10 years may be required to identify policy trends and watersheds that influence the trajectory of sport policy. Despite the slipperiness of the concept, it is possible to take a more pragmatic approach to periodisation within this study, based on significant events or moments that prompted sport policy change and provide distinctions between time spans, as illustrated in Table 4.6.

The identified time periods relate to periods of stability and continuity, following a particular watershed, usually a change in government or PM, which have impacted on government/UKS/SE relationships with, and expectations, functions and operations of NGB/SMNGBs. Indeed, it could be argued that the salience of sport to government was a new menalité that embraced Labour, Conservative and Coalition political thinking of sport as a tool for social good, which was not present pre-1960 when sport was ‘almost the quintessential voluntary activity’ (Holt and Mason, 2000, p.146). The final period identified began in 2015 and it is acknowledged that this may well extend beyond the end date for this research in 2018.

103 Table 4.6: Periodised Timeline

Periodisation 1960-1995 1995-2000 2000-2010 2010-2015 2015-2018

Justification Labour’s Welfare

This chapter has provided discussions and an overview of the research strategy, and the methods considered to be the most logical and appropriate, in relation to the underlying ontological and epistemological assumptions of this research, the achievement of the research aims and objectives; and answering the research questions within this study. The directional relationship between ontology, epistemology, methodology, methods and sources presupposes a logical link between the above constituents of research. This implies that ontological and epistemological assumptions impact on the choice of methodological approaches, which equally have a bearing on the selection of research methods within any given

104 study. It is therefore argued that this research is judged on how its constituent parts logically link together, as endorsed by Grix (2002).

A critical realist position has been adopted for this study, the assumptions of which are that both observable and unobservable social and political phenomena exist independently of our individual beliefs, structures constrain and facilitate rather than determine outcomes, and causal explanations and meaning of social phenomena require an understanding of the dialectical relationship between structure and agency (institutions and causal groups), within broad historical, political and social contexts of which the phenomena are located. Critical realism supports an intensive qualitative multiple case study approach to investigation, and the use of document analysis and interviews as methods for data collection. Within this research, three small to mid-sized NGBs of Olympic sports in England were selected as case studies, namely, England Handball, Table Tennis England, and Volleyball England.

The aim here being to analyse the complexity, diversity and uniqueness of each social entity, their perceptions, beliefs, values and ideas, and the role of structures in shaping them (Phillpots et al., 2010). The relationship between SMNGBs and sport policy has been treated theoretically through the use of the ACF, to analyse sport policy and provide competing explanations to understand the effect of sport policy on the structure of SMNGBs, their external relationships with government and its agencies (UKS/SE), and their position as ‘policy shapers’ or ‘policy takers’ within the UK sport policy domain. This links with the critical realist notion of retroduction.

The choice of document analysis and semi-structured interviews as the primary methods of data collection, were considered strategic and appropriate to answer the research questions, complementary and comparative (Mason, 2002), as well as having the greatest potential to yield corroborative findings that were ‘convincing and accurate’ (Yin, 1994, p.92), and develop an empirically and theoretically grounded argument (Mason, 2002). The semi-structured interviews with key informants, together with the analysis of key documents (e.g. strategic documents, WSPs, annual reports, and sport policy documents) invited useful insights into the beliefs, interpretations, perceptions, opinions and judgements in relation to the relevant themes of governance, funding, capacity building, participation, and partnerships.

The themes selected were considered to be dimensions of operational activities

105 linked to mechanisms for delivering desired policy outcomes, and determinants of policy shaping or policy taking.

A periodised timeline was used to identify time spans as periods within which sport policy directives changed or remained stable under particular political ideologies.

The significance of which was to illuminate the impact of government and governmental agencies on SMNGBs, and vice versa. The employment of retroductive thematic analysis offered ‘an accessible and theoretically flexible approach to analysing qualitative data’ (Braun and Clarke, 2006, p.77), where the initial case study was analysed to provide greater insight and understanding of the theoretical concepts for subsequent case study analysis, and thus allow theoretical generalisation. The emphasis on data triangulation and the precise documentation of the data base and maintenance of a chain of evidence, providing a demonstrable and attainable degree of validity and reliability for the reconstruction of the study from research question to conclusions. Document analysis, thematic analysis and cross-case analysis of similarities and differences between the data yielded from the different sources and the theoretical policy framework, building a picture of the causality between structure and agency, and therein the extent to which SMNGBs are primarily ‘policy shapers’ or ‘policy takers’, in line with the critical realist paradigm adopted for this study, and as demonstrated in the empirical chapters that follow.

106