La XVII Asamblea del PRI

Michael Stausberg and Steven Engler

It is generally agreed that methods , together with theories, concepts and categories , ¹ are foundational for modern science: knowledge accepted as ‘scientifi c’ must be based on empir-ical materials ( data ) gathered by using methods that are accepted as ‘scientifi c’, and their analysis must proceed following rules based on ‘scientifi c’ methods by engaging concepts and theories accepted by the respective academic community. Scholars’ dreams, for example, are not accepted as scientifi c data; allegorical interpretation of such dreams is not accepted as a scientifi c method; illumination is not generally accepted as a scientifi c category; and astrology is not accepted as a scientifi c theory. ² Of course, rules for what qualifi es as scientifi c data, methods, categories and theories are subject to change. The discussions and critiques that motivate such change are a basic task of scholarship. In addition, scientifi c data, fi ndings and theories are constantly challenged by non-scholars. Conversely, the borderlines between what is science and what is non-science, or pseudo-science, are matters of ongoing debate and negotiation.

The past decades have seen vivid debates about conceptual and theoretical issues in the study of religion\s. The very concept of ‘religion’ has been challenged as a valid theo-retical category; feminism, postmodernism, poststructuralism, postcolonialism, philosophy of language, evolutionary theory, the cognitive sciences and other intellectual developments have raised a number of epistemological, semantic and methodological issues (i.e. questions regarding the nature, construction, foundations and production of knowledge and meaning);

there has been a wave of new theories of religion (Stausberg 2009). So far, however, these debates have remained curiously distant from issues of methodology.

The neglect of method in the study of religion\s

Issues of research methods are seldom addressed at conferences. Very few articles on methods have been published by leading journals, even in the one that has ‘method’ in its title. ³ Methods are rarely discussed in introductory textbooks ⁴ and separate courses on research methods are seldom included in religious studies programs. ⁵ In this respect, the study of religion\s stands in marked contrast to other disciplines, which put great emphasis on training in research methods—often in the fi rst year—and which have a strong record of published work on methods, including journal articles, handbooks and specialist volumes. There are no

discussions in the study of religion\s that can compete with the level of technical sophistica-tion established in many other disciplines. ⁶ The present volume is intended as a signifi cant step toward putting research methods more fi rmly on the agenda of the study of religion\s, especially for graduate students.

There are several reasons for the general neglect of research methods in the study of religion\s. A major one is the fragmentary situation of our research landscape, in which some scholars learn textual methods while others become familiar with qualitative social inquiry as part of their training. This relates to the often-heard claim that the study of religion\s is different from other disciplines because it has no research methods of its own. The fact that the fi eld has no sui generis methods is true, and obviously so: almost no discipline does. The analysis of compositions in music comes to mind as a distinct method, but even the study of music uses a range of common methods such as historiography and source criticism or fi eldwork. All academic disciplines use a wide variety of methods, most of which they share with others.

Fieldwork has never been the exclusive domain of anthropology, and sociologists are not the only scholars who conduct surveys. It is therefore a misconception to think that the study of religion\s is signifi cantly different from other disciplines in its use of a variety of methods; what is different is the scarcity of explicit refl ection on methods in the study of religion\s.

Conversely, one often hears that the study of religion\s is a multi- or pluri-methodological discipline. Again, this amounts to merely stating the obvious. How could it be otherwise?

Given the complex nature of most of the things, facts or affairs that are studied in the humani-ties and the social sciences, there simply are no disciplines that could afford to rely on one method only. There is, in fact, a general consensus (at least outside of the study of religion\s) that different methods should, if possible, be combined in order to achieve stronger results. A multi- or pluri-methodological approach is far from an anomaly. What is anomalous is the implicit conclusion derived from this insight, namely that issues of methods do not require attention (because there is no one method anyway). Actually, unless one mistakenly identifi es methodological pluralism with methodological laissez-faire and dilettantism or with the belief that all methods are equally good for all purposes, one would anticipate precisely the opposite conclusion: that the challenge of having to work with and train students in a variety of different research methods requires substantial and explicit attention and commitment to issues of research methods. Given the nature of the discipline of the study of religion\s, then, we would expect to fi nd a deep and abiding interest in training students in methodology, in refi ning research methods and in methodological creativity. This is clearly not the case.

Rather, method use in the study of religion\s continues to be relatively unsophisticated and surprisingly uniform. It is time for this to change. For a relatively well-established discipline such as the study of religion\s, it is more than a little embarrassing that the present volume appears to be the fi rst handbook of research methods ever published in English. ⁷

Methods

As indicated above, the present volume starts from the basic assumption that methods are the rules of the game in scholarly work. Resonating the Greek etymology of the word (from meta

‘after’ and odos ‘way’), the concept is understood here as a metaphor to refer to a (planned)

‘way’, a specifi c way of doing things, an organized procedure. A scientifi c method, in very broad terms, is the generally accepted mode of procedure in the sciences in a broader sense (including the humanities). In the light of theories, methods construct, collect and/or generate the data for scholarly work. Data are not simply ‘out there’, independent of the observer and the observation. There are no data without methods and theories. Methods help us to analyze

reality but, at the same time, they, in part, produce the data that are to be analyzed. In that sense, by partially producing the realities they then go about to analyze, methods are performa-tive (Law 2004: 143). Methods, and the concepts that inform them and describe them, also have a history, changing across scholarly generations (see Platt 1999: 44–52). The goal of this volume is to give a sense of current methods and discussions of method in the study of religion\s.

It is clear that some methods are more helpful than others (for given purposes and within given contexts), that different types of scholarly work make more productive use of some methods than others, and that all methods impose limited perspectives and select empirical materials (data). These obvious facts, however, do not make methods dispensable, for there is no scholarly (or scientifi c) work without method(s). This should not be misunderstood as saying that there is one special method that guarantees success, that following a method guar-antees success, that methods are beyond critique, or that established scientifi c methods are the only way of obtaining relevant knowledge (though scientifi c methods are the only generally recognized way of obtaining scientifi c knowledge). Methods are not a straitjacket; they allow for creativity and new vision. Not everything can be planned out, of course, and not every plan can be put into practice. In fact, scholarly work is often steered more by external constraints, by improvisation and by bricolage than by a master plan. While methodological competence will yield solid work, brilliant work is often the result of serendipity. Creative scholarly work does not go against method, but creatively uses methods; as all good tools, methods are refi ned in use: some wear out and are replaced by others; some are broadened or reoriented in light of their limitations or when facing the threat of perceived methodological hegemony or imperialism (Law 2004).

In a formal sense, research methods are techniques for collecting and analyzing, or enacting (Law 2004) data in scientifi c or scholarly research. While there is always some degree of improvisation, these procedures or techniques typically follow a plan, a routine or a scheme.

These established procedures should not be misunderstood as immutable laws, but as guide-lines and examples of established or best practice (which is not to deny the dynamism of practices). Accordingly, the present volume does not intend to regulate or standardize research practice in the study of religion\s but to improve research and to stimulate its further develop-ment by providing refl ection and suggesting alternatives.

Some key methodological issues

The application and discussion of the underlying principles of these procedures is called methodology. Methodology refers both to general technical issues regarding methods (i.e., case or sample selection, data collection and analysis), and to the theory and conceptualiza-tion of methods. We will address each of these in turn.

Research design

The fi rst, technical sense of ‘methodology’ incorporates several issues: e.g. research design;

relations and tensions between qualitative and quantitative methods; selection of methods;

and means of validating results, including the use of different methods in conjunction.

Research design is covered in a separate chapter in this volume (see Chapter 1.5 ). Here we will confi ne ourselves to listing some of the basic steps involved in effective research design: ⁸ • Identifying the core research question or problem and the series of specifi c questions or hypotheses that will investigate, support or elaborate that core issue (What is the goal of

the study? What lacuna is it meant to fi ll? What motivates the desire to generate knowl-edge on this particular issue? What more specifi c issues will serve as stepping stones to generating that knowledge?).

• Reviewing the relevant literature (What has been published that is comparable in terms of substantive focus, methods and range of theoretical approaches used with similar issues and materials? What will the proposed study add? Are there experts in the area with whom one can consult?).

• Choosing a basic strategy (What sort of study seems best suited to addressing the core research question [comparative, retrospective, longitudinal, case-based]? Does a qualitative, quantitative or mixed-methods approach seem more promising? [see also below]).

• Deciding on the place of theory (e.g. Is the choice of theory established from the start, as when applying or testing a theory, or will one or more theoretical frames emerge during analysis?).

• Specifying methods of data collection and analysis (Which precise method(s) will be used?

How will this/these address the research question and hypotheses? How will a case or sample be chosen, and how does this choice relate to other questions on this list? Is a pilot study or a pre-test of the data collection instrument(s) warranted? How will data be managed?).

• Dealing with logistical constraints (How will limited resources of money, time, assistance be distributed?).

• Assessing the value of the results (To what extent are the results of the study repeatable?

What is the integrity of the results? Are the conclusions applicable to other contexts?

[these issues are addressed in greater detail below]).

• Identifying ethical values, issues and potential problems (Is the topic researchable or is it too sensitive? Are any additional ethical principles and/or risks potentially involved in the research? What is the relevant institutional procedure for obtaining ethics approval?

[this issue is addressed in a separate chapter on research ethics in this volume—see Chapter 1.6 ]).

• Planning ahead for dissemination (How will the results be presented for peer review?

What genre(s) of scholarly presentation/writing will be used? Will these choices impact other aspects of the study, e.g. requiring especially refl ective fi eld notes in order to write in a more refl exive ethnographic style?).

Quantitative and qualitative methods

One of the most signifi cant of debates in methodology concerns the use of quantitative versus qualitative methods. To simplify, quantitative methods employ numerical measure-ment while qualitative researchers do not. Disagreemeasure-ments refl ect basic positions on research design: some advocates of qualitative approaches argue that certain things are simply not amenable to measurement; while some advocates of quantitative methods criticize the subjec-tive nature of qualitasubjec-tive work. ⁹

However, it has also become clear that there is signifi cant overlap and that the distinction sometimes breaks down. Quantitative and qualitative methods are often combined in mixed-method research designs. ¹⁰ Beyond the sociology and to some extent the psychology of religion, quantitative approaches are currently not very popular among scholars of religion, who appear, as a group, somewhat biased against such approaches. However, one should not ignore the crucial interpretive dimensions of quantitative work such as the construction of variables to measure concepts—think of the intricate question of how to

measure ‘religiosity’. Moreover, scholars of religious groups—in particular of marginal ones—often neglect to provide elementary quantitative data, which would be very useful for other and later scholars (Pitchford et al. 2001). While there are some prototypical quantitative or qualitative methods such as statistical surveys (quantitative) and unstructured participant observation (qualitative), one fi nds quantitative and qualitative varieties within the scope of most research methods (e.g. content analysis, discourse analysis, interviews, etc.).

The distinction between these two types of methods remains widely used, but it is inter-preted in different ways. At best, the distinction may be useful to distinguish tendencies and general perspectives on research interests and strategies:

• quantitative methods are often more focused on precision (e.g. closed-ended or categorical questions), qualitative methods on richness (e.g. open-ended questions);

• quantitative methods are often more concerned with generalization, qualitative methods with description;

• quantitative research is often more structured (e.g. emphasizing the use of data collection instruments), while qualitative research is more fl exible (e.g. emphasizing the selection of observation sites);

• quantitative methods tend to address relations between variables, while qualitative methods tend to investigate the meaning that individuals and groups ascribe to human or social phenomena;

• quantitative methods often have a more distant relationship to their objects, while qualita-tive methods presuppose a closer relationship to their subjects;

• quantitative methods generally produce results that are amenable to statistical analysis, while qualitative methods necessarily use less formalized techniques, for example in potentially assessing the representativeness of samples or the validity and signifi cance of results; and • quantitative methods are generally associated with very structured forms of scholarly

writing in the dissemination of results (including literature review, method(s), results and discussion/conclusion), while qualitative methods are associated with a much wider, more fl exible and often creative range of genres and styles.

Criteria of excellence

Three well-known criteria for evaluating the goodness of research data are their reliability, validity and generalizability . In a general sense, reliability refers to the consistency or stability of data or measure of a concept; validity refers to whether an indicator (or a set of indicators) accurately refl ects (or measures) the concepts it was designed to refl ect or, alterna-tively, whether it accurately predicts relevant outcomes; and generalizability refers to the applicability of fi ndings beyond the sample of a given study.

The literature distinguishes between different forms of reliability and validity, and meth-odological research has developed methods for judging and improving the achieved degree of these criteria of excellence (at least for some methods). This has led to the elaboration of various fi ne-tuned conceptual distinctions. These are best addressed in relation to specifi c sets of methods in given frameworks of studies (i.e. research designs).

These concepts raise important issues that are relevant when conducting and assessing research. Regarding reliability one might ask, for example:

• To what extent would the fi ndings be different had the data been collected at a different date, with a different sub-group, or based on different source material?

• Would two or more observers or interpreters come to the same, or at least similar, results when looking at the same data (and would two or more researchers have produced the same, or at least similar, data in the fi rst place)?

Regarding validity one might, for example, pose the following questions:

• Are the constructed data suffi ciently relevant and specifi c for the object of study?

• Would fi ndings still be accepted if other relevant theoretical criteria were applied?

• To what degree do the fi ndings relate to people’s ordinary lives beyond the context of the study itself (‘ecological validity’)?

• Would the analysis stand if data collected with other methods were considered? (E.g.

would the analysis of a ritual based on a philological analysis of the textual sources yield the same interpretation as one based on participant observation? Would an interpretation based on interviews with the main actors yield the same interpretation as the interviews with observers?)

• To what extent can fi ndings be generalized across the social settings studied (‘external validity’)?

Regarding generalizability one might ask, for example:

• Is the chosen case or sample suffi ciently representative, typical, exemplary or compelling that the fi ndings are likely to apply to relevant broader groups?

• To what extent and in what ways does the analyzed case fi t with other cases? Can it be translated to other cases?

• To what extent is the analysis relevant for broader (systematic and theoretical) issues in the study of religion\s?

• What degree of generalizability is appropriate or desirable (from sample to population, or even more broadly to similar populations)? Is this a micro-study that addresses only a particular case?

Obviously, questions such as these are pertinent to research in the study of religion\s. Scholars have suggested several strategies for increasing the validity of research. For engaged or ideologi-cally committed scholarship, fi ndings can be validated in a catalytic manner, i.e. if the study helps the concerned people to improve their situation. In mainstream scholarship, a central form of validation is academic communication, for example by discussion and peer review.

Another more specifi c strategy is known as respondent (or member) validation: in order to receive feedback, scholars present their fi ndings to the people with or about whom they have conducted research. This strategy (which can be practiced in various forms and is only available when studying living groups) can be used as a way to corroborate research fi ndings, as a further step in the collection and analysis of data and/or as a way to enhance refl ections on the research process by the researcher. While this strategy can save one from errors and generate new insights, corroboration by the subjects (which can sometimes be unintentional, such as when informants implicitly confi rm an interpretation by rejecting it) is not itself suffi cient to validate research. Subjects may not understand the scholarly terminology and mode of discourse and may not even have the time to read long texts and—perhaps not only for them—often tedious

Another more specifi c strategy is known as respondent (or member) validation: in order to receive feedback, scholars present their fi ndings to the people with or about whom they have conducted research. This strategy (which can be practiced in various forms and is only available when studying living groups) can be used as a way to corroborate research fi ndings, as a further step in the collection and analysis of data and/or as a way to enhance refl ections on the research process by the researcher. While this strategy can save one from errors and generate new insights, corroboration by the subjects (which can sometimes be unintentional, such as when informants implicitly confi rm an interpretation by rejecting it) is not itself suffi cient to validate research. Subjects may not understand the scholarly terminology and mode of discourse and may not even have the time to read long texts and—perhaps not only for them—often tedious