A Multi-Agent System in Ambient Intelligence for the Physical Rehabilitation 117
Fig. 1. Snapshot of the user interface to define a therapy, using Microsoft PixelSense
The therapist can easily instantiate the metamodel by using a user interface devel- oped for Microsoft PixelSense (see Fig 1) and specify the movements that the older person has to perform by using Microsoft Kinect. The resulting model can be inter- preted by the system which automatically generates therapies that are performed by older people with motor impairment problems. These therapies are also supervised and monitored by the system using MS Kinect and other kind of sensors that provide information about physical conditions of the older person in a transparent way. Fur- thermore, in rehabilitation context, an older person only has to move according to the specified therapy without direct interaction with any devices, taking advantage of the AmI paradigm. Moreover, the therapist defines a set of Fuzzy Inference Systems (FISs) that allows our system to adapt the therapies and to decide the performance order of the tasks, activities and therapies at runtime. In order to support these fea- tures, a MAS has been integrated into our system, which will be described in the next section.
118 C. Roda et al.
be either gestures or postures). Thus, this MAS is designed to achieve several goals:
(1) it supports the execution of the therapies designed by the therapist for older peo- ple, (2) it carries out sensoring activities to monitor how well an activity is being done. Furthermore, this MAS also provides monitoring for some ambient features, namely the oxygen level, the stress, the beats per minute and the current mood of the person. All these features are captured by means of dedicated hardware devices that provide the data to the MAS.
Next, a description of the overview of the design of our MAS is included.
3.1 MAS Development Methodology
The design of our MAS was made by using Prometheus Design Tool [27], the tool for supporting Prometheus methodology for intelligent agent development. Prometheus comprises three main stages: System Design, Architectural Design and Detailed De- sign. System Design is considered a stage to help in organizing the requirements for the MAS by expressing them into a comprehensive graphical notation. This phase is made based on the principles of the goal-orientation approach. In Architectural De- sign stage the composition of the MAS is designed. This stage includes deciding what agents the MAS will have, the messages they will exchange or the capabilities they will exhibit. Finally, Detailed Design stage is related to the internal design of agents.
As these stages are made for all the artefacts involved in the MAS, the System Over- view diagram is composed. This diagram is a great tool for representing the general architecture of a MAS system (see Fig 2). This graphical notation includes the agents involved (in light brown color), the protocols including the messages they exchange (in magenta color), the actions each agent is responsible for (in light green color) and lastly, the percepts that represent the external information arriving to the MAS (in light red color) from the ambient.
3.2 Managing Ambient Information
In Fig 2, the overview of the design of our MAS system is depicted. To manage the external information arriving from the ambient to the MAS, dedicated agents are used. We decided to use one agent for each type of external ambient information (percept), because it is easier to extend its capabilities. This is an important feature, since the availability of sensors for assorted sources of information advances very quickly. Thus, our MAS must be ready to easily consider extra valuable infor- mation from the ambient to improve the available information used in the decision making process. Therefore, we have an agent for each type of information arriving to the MAS from the ambient. The data sources we are using are oxygen level, posture, gesture, stress, BPM and mood. AgentOxygenLevelCapture, AgentPostureCapture, AgentGestureCapture, AgentStressCapture, AgentBPMCapture and AgentMoodCapture are the agents responsible for capturing the information from the ambient, respectively.
A Multi-Agent System in Ambient Intelligence for the Physical Rehabilitation 119
Fig. 2. MAS design overview
120 C. Roda et al.
Oxygen level represents the saturation of oxygen in the blood, and it is a good in- dicator to help to find out when the person making the therapy is becoming tired.
Posture represents the different postures that the person is making to do the therapy, e.g., raising the arms or the elbow. Devices like Microsoft Kinect [28] are used to capture these postures, which provide information about how well a person is making each step of the therapy. Some therapies require more fine-grained movements, such as gestures made with the hands. Devices like Leap Motion [29] are able to detect gestures made with both hands. Stress level is also used to avoid making the therapy too stressful for the person. Beats per minute (BPM) is also a good measure for fa- tigue. Finally, mood change is currently also supported to find out if, for example, the person is happy, excited or pained when making the therapies. Please, notice that in many cases the values reported by the ambient are relative. Therefore, the historical data and some statistics are required to make a proper use of this information.
All these information must be stored and managed. To store and manage all the in- formation, all the agents responsible for capturing the information send their pro- cessed data to the AgentPersonDataManager. This agent is responsible, on the one hand, to integrate all the arriving information into the knowledge bases and, on the other hand, to dispatch this information on demand to any other agent that requires this information. Thus, an up-to-date and consistent version of the data is supplied to every agent so as to make more precise decisions.
3.3 Therapy Execution
The execution of the therapy is controlled by AgentTherapy. This agent is responsible for running the therapy. Therefore, it will control the execution of the activities, their tasks and the steps these tasks prescribe. Therapies are initially designed by the thera- pist, but they can be adapted at run-time according to the input information from the ambient. The transition from one element of the therapy to the next is an important fact in the specification of the therapy. For example, if the person is tired it will be better to reduce the number of repetitions for each element. These transitions are specified by using fuzzy rules. Fuzzy rules are also created by the therapist, which provides a natural way to express how transitions should be made by using linguistic values rather than numerical values. Unlike the classical logic, the fuzzy logic does not establish rigid boundaries when dealing with a variable (e.g., Number of repeti- tions is low when it is 3 or less), but it defines regions in the domain of the variable that can overlap and determines the degree of truth of a fact according to the real val- ue. Those regions are known as Fuzzy Sets and the degree of truth of a fact is the degree of membership of a crisp value of the variable to the fuzzy set referred to in the fact. The degree of membership is calculated by using a membership function that defines that fuzzy set. The fuzzy sets are tagged with Linguistic Values. All fuzzy inference is handled by AgentInference. A specific agent is used for the inference engine aiming at keeping the inference as separated as possible from the rest of the agent. Thus, we support the isolation of possible changes in the inference engine.
Furthermore, in this way, we have also a single agent that access the profile of the person. This is also interesting from the point of view of privacy, because by securing
A Multi-Agent System in Ambient Intelligence for the Physical Rehabilitation 121
a single agent, the privacy of the data of the persons is improved, thus reducing the development effort for privacy issues. Privacy has been proven an issue for any kind of medical related software, since most people find their medical information espe- cially private.
3.4 Logging Therapy Statistics
Another action carried out by AgentTherapy is generating statistics. For therapist to follow the progress of their patients, they need to get feedback about the execution of the therapies they design [30]. Aiming to provide such feedback to the therapist, AgentTherapy logs every action the patient does during the execution of the therapy, logging every activity, task and step the patient took during the therapy. These statis- tics include currently completion time, repetitions, postures and gestures identified, tries before a posture or a gesture is made ok, and fatigue. With these statistics, the therapist can follow how the patients progress and they will provide valuable feed- back to design other therapies or adapt the current ones.
3.5 Adapting Therapies
Although the therapists have feedback from the logging to try to design the therapy, it is impossible to foresee all the ambient factors that can have an influence during the execution of the therapy. Therefore, the transitions from one element of the therapy to another is made in terms of fuzzy rules. These fuzzy rules support the therapist in designing under what circumstances the transition should be to one element or anoth- er, thus supporting and adaptive workflow of the therapy execution. These fuzzy rules are evaluated by the AgentInference agent.
On the other hand, therapy execution can also be adapted in another manner.
In this case, AgentAdaptation makes use of the information gathered by AgentPersonDataManager from the information captured through the sensors and the therapy execution logs to adapt the fuzzy sets to reflect the issues detected. For in- stance, if TherapyAgent reports that the user has made too many repetitions of a ges- ture or a posture to do it right, AdaptationAgent decides to reduce the precision re- quired to consider the posture or gesture as right by adapting the corresponding fuzzy sets. This loss of precision can be because of fatigue, therefore AdaptationAgent, by checking the evolution of the data captured by the sensors, can decide to recommend a break in the therapy execution to the patient. Thus, we provide a therapy execution environment that can cope with a vast number of ambient situations that can change the initially predicted therapy execution defined by the therapist.