In the preceding case study, Cummins et al. had already obtained what they took to be considerable confirmation for their hypothesis from their field experiment. So it is natural to wonder why it was worthwhile to introduce an abstract computer simulation as a way of proceeding with the research program. What benefit did the computer simulation offer, given that the researchers already took their hypothesis to be well-confirmed by their empirical study? Unlike the reverse-engineering simulations I discussed in Chapter 2 – in which a simulation amounts to a last resort for investigating a target system about which few discriminating
87
observations are available – this was not a case for which the researchers lacked a testable hypothesis. Cummins et al. not only had a promising hypothesis (that the children's errors on the word problems were due mostly to inadequate story comprehension and not inadequate mathematical understanding) formulated in advance of conducting the simulation, but they had also already amassed a good deal of confirmation for their hypothesis by performing a traditional empirical study with the target population. The motivation for choosing to run a computer simulation of the target system is therefore much less obvious than it is in the case of reverse- engineering simulations.
It is important to note that the researchers did take this simulation study to have a confirmatory function. One of the stated purposes for employing the simulation was to test the hypothesis favored by Cummins et al. against the rival hypothesis “that comprehension failures reflect deficiencies in [logico-mathematical] conceptual knowledge” (1988, p. 419). Certainly, the researchers had marginally more evidence for their story-miscomprehension hypothesis (and against the rival hypothesis) after running the computer simulation than they did on the basis of just the empirical results. But, importantly, it does not seem that they had more confirmation than they could have had with, for example, additional empirical research. It is therefore illuminating to consider why the simulation in this case is a particularly valuable complement to the empirical work.
One might reasonably think that a confirmatory benefit of the simulation over the empirical study is that the computer simulation enabled the researchers to formulate and test a more precise hypothesis than they were able to in the empirical study. As I mentioned above, the researchers tweaked the simulation's language processing modules in ways that were suggested by other psychological research on children's language processing (e.g. the researchers
88
reprogrammed the simulation to understand “some” as an adjective rather than as a quantifier). However, in practice, the researchers acknowledge that these precise tweaks were not clearly confirmed, even though they took their general hypothesis to be confirmed. The researchers made specific predictions about the ways their modifications to the language-processing module would alter the simulation's performance, and the simulation produced a number of discrepancies with these predictions (Cummins et al. 1988, p. 427). For instance, there were some problems that employed the term “some” that children frequently failed to answer correctly. The researchers expected that the children were interpreting “some” as an adjective, and that this was a common cause of the failure to answer this question correctly. However, the computer simulation did not reflect this; the program consistently answered these questions correctly, even when it interpreted “some” as an adjective (p. 427). The researchers reported that it was “not clear why this discrepancy occurred,” but they followed their discussion of these discrepancies with this overall assessment of the confirmatory value of the simulation:
Most important to our endeavor is the fact that the patterns of solution difficulty reported here and elsewhere in the literature could be accounted for simply by manipulating linguistic aspects of the simulation program. The major determinant of its solution characteristics was whether the nature of its linguistic processing afforded access to its conceptual knowledge. (1988, p. 427)
So while the researchers' general hypothesis (that children's errors were due to inadequate story comprehension rather than a deficient grasp of mathematical concepts) was incrementally confirmed by the relative success of the simulation that featured a weakened language processing module, it is not actually clear that the simulation offered much by way of clear confirmation or disconfirmation regarding any precise hypotheses about the particular features of language processing that accounted for the children's error patterns.
89
But even if the researchers had been able to confirm a more precise version of hypothesis by implementing this simulation, it is clear that they took the simulation to have a further value. They mention at several points that the simulation is particularly useful for explaining the target phenomenon. In their discussion of the results of their study, Cummins et al. explore ways in which their simulation results support much broader claims than the hypothesis about children's problem solving that motivated the study. They devote time to characterizing the simulation results at a general level, reporting for instance that those results “suggest that common solution error patterns are directly related to the linguistic sophistication possessed by the solver” and that “robust linguistic knowledge produced successful solution attempts regardless of text characteristics, but the attempts were more time and resource-demanding” (p. 435). After describing their findings at this level, they go on to suggest that:
This description of successful arithmetic problem solving... is consistent with descriptions of problem solving in other “adult” domains, such as physics (e.g. Chi, Feltovich, & Glaser, 1981) and chess (Newell & Simon, 1972). In these domains, expertise development is typically considered a matter of constructing problem type schemata and using such structures to guide comprehension (p. 436).
The researchers describe this capacity for problem solving via identifying problem-type schemata as arising from the characteristic interactions of the modules featured in the simulation. Specifically, they write that the particular ways in which the linguistic processing module facilitates access to the relevant logico-mathematical concepts makes this construction of problem type schemata possible (p. 435). Furthermore, they suspect that the explanation of problem-solving provided by the simulation generalizes, writing that:
The implications of the present paper go far beyond the domain of children's arithmetic. The various determinants of problem difficulty that were explored here are likely to be important in any domain where problems are described in verbal formats. (1988, p. 437)
90
Thus, though they originally set out to explain why word problems are so difficult for children, they note as a virtue of the simulation that it appeals to features of problem solving that are also cited in explanations of problem solving in demographics beyond the children they worked with in their empirical study. I take this to be evidence that the researchers see as one major benefit of the simulation that it provides a better explanation of the target phenomenon than it would if it were only locally applicable. The sense in which it is not merely locally applicable is that the modules out of which the simulation is composed have analogues in various other human problem-solving systems. Cummins et al. appear to suggest that simulations consisting of the same basic “ingredients” (where sameness is assessed at a more general level of description than a precise description of the program's content) could be developed to offer explanations of other patterns in problem-solving.
I will argue in Section 5 that the fact that this simulation accurately represents not only its target system, but also additional target systems within the same domain, is an important explanatory virtue, and one that computer simulations are particularly well-suited to promote. In terms familiar to philosophers of science, computer simulations are often uniquely well-suited to provide unificatory explanations of empirically observable phenomena. I will offer an account of this explanatory achievement in Section 5.