The aim of Study 4 (pilot study) was to characterize difficulties, if any, in trend interpretation by asking participants to look at and then describe a real-world time-series graph that contained an underlying long-term trend as well as substantial short-term variability. Asking people to describe what they think a graph shows can identify which information is salient and encoded (Hegarty, 2011; Shah & Carpenter, 1995).
To see if people correctly identify long-term trends from time-series graphs that also show significant short-term variability, verbal descriptions were collected from individuals exposed to a real-world graph showing such
characteristics. The graph chosen (Figure 10) shows data for Northern Hemisphere spring snow cover extent between 1922-2012, published by the Intergovernmental Panel on Climate Change (IPCC, 2013a) which is one of the figures from the AR5 Working Group 1 SPM. The data indicate a significant downward trend over the whole time-period, together with substantial inter- annual variability. In the text of the SPM, the authors indicate that snow cover extent has decreased since the mid-20th century (IPCC, 2013a), suggesting that this message is an important communication goal. Given that the short-term variability is explicitly represented in a complex line graph, whereas the long- term trend is not, it is predicted that the majority of individuals will describe short-term variability, but may not describe long-term trends.
Method
Participants
Twelve undergraduate students (10 female, two male) from the University of East Anglia took part in the pilot study in return for course credit or a nominal
payment. Their average age was 21 years (range 19–29 years). None of the participants were studying environmental sciences.
Apparatus and Materials
The target stimulus consisted of Figure SPM.3a from the IPCC SPM (IPCC, 2013) (Figure 10). The stimulus were presented on a TFT LCD monitor (51cm x 29cm), set to 1280 x 720 pixels. Eprime Version 2.0 (Psychology Software Tools Inc., Sharpsburg, USA) was used to control stimulus presentation and record data. Verbal responses were captured via a headset microphone.
Procedure
Participants were instructed that on each trial, they would be shown a graph or diagram to study and would then be prompted to “describe what you think the graph is trying to show”. The trial in the present study was presented to
participants embedded within another study (Study 5). Participants were therefore shown a visual prompt indicating that they should study and prepare to describe the next graph they see. The graph was presented for 15 seconds, during which participants simply looked at the figure. Participants then saw a ‘Now describe’ prompt and the same figure re-appeared on the screen. The figure remained on screen until the participant completed their verbal response (indicated by pressing the spacebar on the keyboard) or until a maximum time limit of 45 seconds was reached (Figure 12).
Figure 12: Presentation of experimental trial.
Coding
Verbal descriptions were coded to assess the presence (coded as ‘1’) or absence (‘0’) of the following characteristics: (a) the data represent changes in snow cover over time; (b) a general downward trend; (c) a downward trend between ~1960 and ~2012; (d) short-term variability/fluctuation. Coding criteria are shown in Table 3. Inter-rater reliability across all aspects and all coding was K = 1.000, p < .001 (i.e. complete agreement).
Table 3. Study 4 coding criteria for the four characteristics.
Characteristic Characteristic phrases used to code (a) the data represent
changes in snow cover over time
Refers to ‘snow cover’ and refers to time in the same utterance, such as ‘over time’, ‘over years’, ‘between ~1900 and ~2012’.
(b) a general downward trend
Refers to the plotted data, as a whole, showing a downward trajectory, such as ‘going down’, ‘decreasing’, ‘decline’ and does not tie this description to a specific time period, or explicitly refers to the whole time period.
(c) a downward trend between ~1960 and ~2012
Refers to the plotted data as showing a downward motion, such as ‘going down’, ‘decreasing’,
‘decline’ and ties this description to a specific time- period congruent with the period of the graph representing ~1960 and ~2012.
(d) short-term
variability/fluctuation
Refers to the plotted data as showing variability, such as ‘peaks and troughs’, ‘goes up and down a lot’.
Results
All twelve participants correctly identified that the data represented changes in snow cover over time (Table 4). Five participants (42%) described some form of downward trend (either a general trend and/or a trend ~1960 and ~2012), whereas seven participants did not describe any form of a trend (58%). Taking those who did describe a trend and those who did not describe a trend as two groups, the likelihood of describing the short-term variability was then compared. Of those who described a trend, only one (20%) also described short-term variability, compared with five of the seven participants who did not describe a trend (71%) (p =. 01, Fisher’s Exact Test).
Table 4. Study 4 frequency of the number of individuals who verbally described each characteristic.
Characteristic Frequency count (percentage)
(a) the data represent changes in snow cover over time
12 (100%)
(b) a general downward trend 5 (42%) (c) a downward trend between ~1960 and
~2012
1 (8%)
(d) short-term variability/fluctuation 6 (50%)
Discussion
These pilot data suggest that when presenting graphs that contain an underlying long-term trend and substantial short-term variability, spontaneous interpretation of the long-term trend is not guaranteed – indeed fewer than half the participants in the study described any kind of trend. Of the participants who did not describe a trend, the majority did describe short-term variability. Conversely, few of those who described a trend mentioned short-term variability. Hence, other than
describing what the data in the graph represented (snow cover over time) which corresponded with the graph title, participants typically only described one aspect of the data (either trend or variability). It’s possible that participants felt that they had to only describe the most salient aspect, rather than all aspects – i.e. it might be that they did encode the other characteristic, but did not mention it. Conversely it might be that participants only encoded the characteristic of the data that they described.
Although these two possibilities are not differentiated in this pilot study, the results are consistent with studies that have found impaired task performance with line graphs that contain a high level of variability compared with graphs with less variability (Correll, Albers, Franconeri, & Gleicher, 2012; Carswell, Emery, & Lonon, 1993). Indeed, the ratio between the strength of the trend (i.e. the angle) and the extent of the short-term variability (i.e. the vertical spread) may be
important in determining to what extent different characteristics are salient, to what extent spatial processing is required, and to what extent task performance is impaired. The ratio between the strength of a trend and the extent of short-term variability can be conceptualised as signal (trend) and noise (variability). Snow cover (as plotted in the stimuli graph in this pilot) has a comparatively low signal- to-noise ratio (Krasting, Broccoli, Dixon & Lanzante, 2013). When data that has a greater signal-to-noise ratio are plotted in a line graph, the connected line may contain fewer visual chunks (by virtue of less short-term variability relative to the strength of the trend) and/or may be encoded as a single line, making the
identification of a trend easier.
While this pilot study does not identify the extent to which participants mentally encode and process trend information relative to variability information, the data do suggest that when asked to describe a complex time-series, trend information may not be salient. Mental representations, as opposed to verbal descriptions, of long-term trends and short-term variability are considered next in Study 5.