SERVICIO CONSULTADO

LinguaBytes is aimed at stimulating the language and communication skills of children between 1 – 4 years old with multiple disabilities through interactive and adaptive educational toys. One part of LinguaBytes involves reading interactive stories in which (new) words are offered to the toddler in an animated visual scene [4]. One of the stories we have created is a simple linear story about a boy and a girl, Tom and Tess, who visit the children’s farm (Figure 3). All scenes are animated and supported by audio of a narrator and sound effects. The toddlers can ‘read’ the story on screen by moving a physical slider from left (first page of the story) to right (last page) over a physical representation of the story (Figure 4). To identify the user of the slider a personal identification tag is inserted in a slot in the side of the slider’s housing.

In the following, we will describe how adaptivity and adaptability play a role for five example users from our target group (Figure 5), in order for them to read the story. We will use fictional names, but want to emphasise that the examples are actual participants of the LinguaBytes project.

Figure 1. PC’s were originally designed for office use

Figure 2. Adaptability: the user adjusts the product and Adaptivity: the product adjusts itself to the user

5.1

Lisa

Lisa is 3;1 years old and has a developmental age (DA) of 2;4. She has hydrocephalus and trouble breathing due to a congenital defect of the windpipe. As an effect, Lisa has diminished vision. Her fine and gross motor skills are good, but usually she sits in a special, tailored chair during therapy. Lisa likes to inspect stuff from up close, picking up things and manipulating them.

5.1.1

Adaptability

As a result of Lisa’s condition, it is important that both the screen and the slider are positioned close to her. The screen should be tilted so that ambient light doesn’t cause too much glare. The slider should be placed right in front of her, fixed to the workspace, slightly to the right so she can handle it optimally. The graphics on the screen should have high contrast and move slowly.

These requirements can all be placed in Aarts’s ‘personalization’ stage. They all involve the physical adaptation of the product in relation to the user within a space (Figure 6). We see this as physical adaptability of the product.

5.2

Dobson

Dobson is 2;11 years old with a DA of 1;6. His fine and gross motor skills are good although he has psychomotor retardation; but he can move around by himself, doesn’t need special furniture and can manipulate objects like any toddler his age. He usually plays sitting on the floor. His problem is that he has an autism spectrum disorder. In his case this means that Dobson

is very reluctant to anything new or irregular.

5.2.1

Adaptability

Dobson would, contrary to Lisa and the following children, play on the floor so the screen should be used horizontally. The slider shouldn’t be fixed to the floor so he could position it himself. However, Dobson won’t use the slider as it is because he doesn’t know it, nor will he pay attention to the story since it’s not about people or situations he knows, but about two colourful drawn figures called Tom and Tess. For him to relate to the story, it would have to be about him, using real pictures. The narrator should be his mother or another safe voice. Clearly, apart from the physical adaptability needed to suit the interaction styles of both Lisa and Dobson, the product should have emotional and cognitive adaptability as well. Moreover, due to the difference in Dobson’s calendar age and developmental age, there is also a need for linguistic adaptability, since for Dobson the story should be less complex than his calendar age would suggest. This accounts for most of the children we describe here.

5.2.2

Adaptivity

To reduce the abovementioned difference between Dobson’s calendar age and developmental age the product should very gradually change its content, e.g. by slightly altering the storyline, changing the characters or making the visual material more symbolic. These changes in content should be very subtle and well monitored. In short, in order for Dobson to develop the product should be highly adaptive.

5.3

Stanley

Stanley is a cheerful boy of 3;1 years old and has a DA of 3;1. Figure 3. Screenshot from the linear story of Tom and Tess

visiting the children’s farm

Figure 4. The linear story slider

He is diagnosed with infantile encephalopathy and has a cerebral palsy (CP) in the form of spastic quadriplegia. This means that he cannot speak, cannot walk on his own and needs special furniture. Also, Stanley has to concentrate really hard to have control over his arms and hands, due to the irregular tone of his muscles. His upper body tends to move involuntarily and his head can suddenly bob sideways. Due to his medication, Stanley drools unintentionally.

As a consequence, in order for Stanley to comfortably interact with our system, more adjustments will be necessary than just re-positioning. Due to Stanley’s diminished control over his muscles, he has trouble letting go of the things he grabs. This means that if he uses the slider, he has difficulties in sliding the short distances between scenes. To avoid Stanley from sliding past the target, the slider will have to limit its range (Figure 7), keeping Stanley in control of the sliding action, but helping him by subtly adjusting its behaviour.

5.3.1

Adaptability

This would mean that when Stanley’s identification tag is inserted in the product, the product would know how to behave by default. In terms of adaptability, this implies that Stanley’s user profile can be set to a ‘spastic quadriplegia/athetosis’ preset, so that the product will know of his disturbed muscular control and behave differently.

5.3.2

Adaptivity

In terms of adaptivity this means that the slider should start reacting to Stanley’s actions. It should contain sensors to detect and monitor Stanley’s behaviour and sense when Stanley has the intention to interact or just can’t let go of his previous action. Using a force sensor in the slider, a small motor to either block it or let it run freely, along with a sensor to detect the position of the slider could do this. At the end of each scene the motor would unblock, Stanley would be able move the slider until it is at the desired position. Then the motor would block

again, fixing the slider’s position.

In the case of Stanley, we can conclude that the focus is no longer just on optimal positioning, but shifts towards the system’s reactions on the user’s behaviour (Figure 8) and Aarts’s use of ‘adaptation’.

5.4

Emily

Emily, a very smart and funny two year-old (her DA matches her calendar age), has spastic quadriplegia mostly resulting in dystonia and athetosis, which cause spasms, abnormal postures and uncontrolled writhing movements.

5.4.1

Adaptability

In the case of Emily it is no longer possible to optimally position the slider, since she continuously moves around. In order for her to be able to use the slider, it is first of all necessary to keep it clearly visible to her at all times, which essentially means that it should be positioned higher and so that it can be freely moved around, for instance by using the system of a balanced-arm lamp (Figure 9). This makes it possible to move the slider within Emily’s eyesight and, when needed, move it towards her hand.

5.4.2

Adaptivity

However, if Emily wants to use the product more autonomously (which would be better for her self esteem) it should automatically follow her eyesight. Additionally, the slider should sense when Emily would try to manipulate it and give her a hand by approaching her hand and willingly slide autonomously upon her touching it. Just like a cat lifting your hand with its nose when it wants to be stroked. For this, it is necessary to be able to preset this in Emily’s user profile (adaptability), and to use sensors and motors again to detect Figure 6. Product and User in context

Figure 8. Relation Product-User through Behaviour

Figure 7. Slider range, adjusted to the allowed interaction

Emily’s behaviour and adjust the product’s reactions. This can also be illustrated by Figure 8.

5.5

Gus

Gus is 3;4 years old with a DA of 1;5. He is diagnosed with infantile encephalopathy and spastic quadriplegia. Gus has the same physical challenges as Emily, along with severe vision problems and epileptic fits.

5.5.1

Adaptability

For Gus, the slider would have to behave as with Emily, but also offer additional auditory feedforward and feedback to help him focus his attention. This behaviour should be set in Gus’s user profile.

5.5.2

Adaptivity

Gus’s regular epileptic fits however, require the product to contain another form of behaviour. Due to his sudden epileptic fits, Gus’s actions are often disturbed, even more than Emily’s. Where even the behaviour of highly athetoid children has some predictability, Gus’s fits are more irregular. These unpredictable mid-interaction losses of bodily control make it necessary for the system to adjust the flow of the interaction. Essentially, during a fit, the product should pause and wait for Gus to recover. When the product senses it has Gus’s attention again, it could continue the scene, or decide to start over and play the scene again.

This kind of adaptivity requires not only that the product continuously monitors Gus’s behaviour, but also try to detect patterns in, or recognize Gus’s behaviour in order to make accurate interpretations and learn for future interactions. This long-term product adaptivity can be seen as Aarts’s ‘anticipation’ key element.

6.

ANTICIPATION, OR LONG TERM