People make use of indicators on a daily basis for making decisions (Acton, 2000; Eder, 2004; Podger et al., 2010). For example, a grey and cloudy sky indicates that it may rain
86 and, thus, that we need to take a thick coat and an umbrella with us. The simplicity of this indicator contrasts with the complexity underpinning the identification of sustainability indicators which aim to assess the interplay amongst environmental, socio-cultural and economic issues (Bell & Morse, 2008; Moldan & Dahl, 2007), or indicators of learning for sustainability which try to assess processes and quality of educational practices regarding sustainability (Vare, 2006).
In general terms, an indicator can be defined as a variable (not a value) which represents an attribute (quality, characteristic or property) of a system (Gallopín, 1997); indicators are images (measurements or observations) of the attributes which describe the system of interest. Sometimes, indicators have also been defined as accessible ‘proxies’ for properties of a system which are difficult to collect, measure and observe, but are important to assess (Eder, 2004). As Meadows (1998) states, indicators show how the system works, what is important in the system and what should be measured. Ideally, indicators have the capacity to identify issues and problems within the system which need to be fixed and improve decision-making processes to solve these problems (UNECE, 2006). However, indicators can also create serious malfunctions and indicate wrong directions for the future when underlying assumptions have not been outlined, data collection methods and reporting mechanisms have not been clearly identified, or guidelines for interpretation are not provided (Reid et al., 2006).
Issues also arise when a reductionist or oversimplistic approach to indicators is adopted which ignores interrelationships or the system itself. Bell and Morse (2008, p. 19) remind us that “a focus on one problem in isolation could at best achieve nothing and at worse create more intractable problems elsewhere.” In this regards, several authors suggest that the systemic nature of sustainability demands whole-system attributes and holistic indicators which can describe the complexity of this dynamic (Gallopín, 1997; Hodge & Hardi, 1997; Sollart, 2005). It has been suggested that sustainability requires a system of indicators which can provide information about each attribute of the system, but also about how the different components relate with each other to produce the overall outcomes (Reid et al., 2006).
87 In the area of education and learning, where my research is located, indicators can be defined as “products or behaviors which serve as evidence that what is expected to be learned is actually being learned” (Eder, 2004, p. 141). This is a difficult task as what is being learned, or the action of learning is sometimes invisible. Thus, indicators will provide critical information about a series of attributes, such as inputs and outputs of learning, which can gauge progress of educational processes (Tilbury & Janousek, 2006). In learning for sustainability, for example, critical thinking is an important attribute. Examples of indicators include:
i) % of formal courses which provide critical thinking as a learning objective in the area of sustainability;69 or,
ii) the number of learners who have benefited from training on critical thinking in the area of sustainability;70 or,
iii) critical thinking is a learning objective embedded in the formal and informal curriculum;71 or,
iv) the learners demonstrate the ability to think critically.72
As observed from these examples, many different indicators can be defined depending on the objectives, assumptions, goals and scale which underpin the assessment exercise. Indicators can be quantitative (examples i and ii) or qualitative (examples iii and iv). The first provide data in the form of numeric values, percentages, rates, or means, and the latter offer information in the form of verbal and written descriptions and observations (Tilbury & et al., 2007). Many times, quantitative indicators have been considered to be more reliable and valuable as they can be easily measured, validated and communicated. This assumption may be true for certain types of indicators and should not rule out the possibilities of developing qualitative indicators (Gallopín, 1997). Some important information related to, for instance, the quality of learning processes
69 Used to assess learning for sustainability within institutional programmes, for example. 70 Used to assess capacity building in the area of learning for sustainability.
71 Used to assess the alignment of course or programme objectives in learning for sustainability. 72 Assesses the quality and/or outcomes of learning for sustainability programme or experience.
88 regarding sustainability cannot be captured through numeric values and quantitative indicators (University of Brighton, 2009).
As an example, the Monitoring and Evaluation Expert Group73 (MEEG) for the United Nations Decade of Education for Sustainable Development (UN DESD) uses quantitative indicators to ascertain general trends and improvements in structures and contexts for learning for sustainability, but admit that only qualitative indicators can provide a genuine indication of progress in this area as these enable processes and quality of learning to be ascertained (Wals, 2009). Tilbury et al. (2007) acknowledge that both types of indicators should be explored and combined in order to address different assessment needs.
A series of questions can assist in interrogating indicators so we can understand their value and limitations: what indicator framework is used?; what are the assumptions underpinning the definition of indicators?; how are the indicators developed?; what is the scale of the indicators?; who is involved in the process?; what is the purpose of the indicators?; who will be using these indicators?; what are the shortcomings of these indicators?; and, how will data be collected and interpreted? It is important to note that whereas indicators are useful tools to guide decision-making and action-taking processes, they are not an “end in themselves” and “should not be the sole basis for judgment” (Reid et al., 2006, p. 6)