All of the subjects managed to communicate a qualitative economic model to DENISE, for example Figure 7.1. However nearly all the models
Subject A B C D E
unique concepts 11% 69% 55% 61% 78%
Table 7.2 Percentage of unique concepts generated
were a single-viewpoint and the one subject (B) that did create a new viewpoint did so more out of experimentation than a purposeful commitment to describe an alternative view.
The positive and negative causal relationships overwhelmed the other relationships in rate of use. Many of the relationships would be conventionally regarded as ‘misconceptions’, e.g. (subject D)
U: unemployment is negatively related to state of economy
This states that a rise(fall) in unemployment would cause the economy to worsen(improve).
The domain concepts entered into DENISE varied widely ranging from traditional economic concepts to politics and even to religion and wars (subject D):
U: religion part_of_+ wars D: Ok go on
U: religion is negatively related to crime
This sort of domain drift is a consequence of exploring the learner’s viewpoint rather than restricting the interaction to a well defined domain. Learners do not necessarily have the same conception of a domain as the system designers. The wide variety of domain topics covered suggest that the models acquired are indeed individualised. Many of these concepts would not be encountered in economics courses. The concepts used by subjects can be compared with concepts used by the other subjects, as in Table 7.2. 55% of concepts referred to by subject C were not referred to by any of the other subjects. Subject A, who had economics experience, concentrated on a small number of well known concepts (e.g. exports, imports, interest rates, unemployment etc) and produced a more tightly connected model.
The unfortunate lack of alternative viewpoints expressed is a consequence of the freedom of the system, if it does not force users into moving viewpoints and they don’t want to then they will remain in the current viewpoint. This is a form of cognitive economy [Scanlon & O'Shea, 1988].
Subject E drew an analogy with supervision of undergraduate students which is reminiscent of peer tutoring reports:
E: I did some demonstrating in biology to some of the undergrads E: and, you know, you’re there doing a higher degree
E: and everything and they’re just at the beginning E: and you talk to them, it gets you thinking E: some of the things they ask you
E: you think , Oh I never thought of it that way
7.5 Summary
Bearing in mind that the sample size for the trials was small the results can be summarised as:
· DENISE can capture individualised models from users · users are forced to examine their own beliefs
· users do revise their beliefs as they interact with DENISE
· domain drift occurs
· users were reluctant to verbalise their reasoning in the experimental situation
· users prefer to remain in a viewpoint rather than move to a new one
· default non-committal responses slow down interactions but may provoke reasoning
Research Questions
In section 4.5 eight research questions were outlined and five were identified as being appropriate to this research:
Yes. The experiences with DENISE indicate that the ISS concept can be used to provoke human learners to examine their own knowledge about a domain. This self-examination, or reflection, is held to be the critical part of the learning by teaching paradigm.
Question 3: what domain-related knowledge does an ISS require? It is clear that an ISS requires some knowledge of the domain. However, this knowledge need not be of the ‘full domain model’ type required by the ITS approach. The DENISE experiment provides support for the view that only knowledge of the type of domain concepts is necessary. The conceptual syntax in DENISE contained no knowledge about the domain of economics yet none of the subjects realised that the dialogue was essentially domain-independent.
Question 4: do learners find ISS interaction useful/interesting? Yes. The subjects, although not explicitly learners in the domain, found the interaction particularly interesting. Without an interest in the domain they did not find the dialogue useful but could see how a dialogue in an appropriate domain might be helpful. All of the subjects thought the role reversal aspect of the ISS interaction was interesting and stimulating.
Question 6: do ISS activities require a concrete task other than teaching?
Maybe. The limitations of the dialogue strategy and the lack of a perceived goal to the interaction were apparent in the post-experiment interviews. A definite goal to the session would have provided focus to the dialogues and possibly prevented the feeling of ‘being lost’ in the domain.
Question 7: does an ISS require an additional student model?
Probably Not. The subjects were not aware of the ‘ignorance’ of the ISS during the interactions and yet proceeded to develop several complicated economics relationships. It does seem to be the case that the system didn’t need to know anything about the learners to provoke a viable dialogue.