The above sections have presented the important key-tasks for building a working prototype of the MECoS concept and being able to perform experiments using this prototype. Of course many more activities had to be performed to build up a complete and fully working simulation environment in which the experiments could be conducted. The following paragraphs give a short overview of those activities (including the prototype and required additional tools for performing the simulation runs and collecting the results). In order to build the simulation environment for the MECoS approach, the following tasks were performed (in this sequence):
Implement an ANN-Test
• implement non-evolving, non incremental variant of ANN in Java as a base (from scratch, the MECM test was not re-used)
• get deeper understanding of how ANNs really work by building an implementation myself (practical experience is required to fully understand the topic )
• look for current java implementations of ANNs and investigate them • a problem was the application of correct activation-functions and parameter values (momentum, learning rate, etc.) of the network to achieve usable learning results
Implement the EMLP (evolving multilayer perceptron) algorithm
• build evolving multilayer perceptron as a first minimal ECoS implementation of the previously built "static" ANN by enhancing/changing the algorithm to an evolving one as described in [110]
• like ecmtest translating the linguistic description of ECoS working scheme into a Java programme code was different
• finding good parameters to get a working emlp is quite difficult, too: even if the algorithm is basically working, parameter tuning is a time consuming task
Implement emlpME (emlp java microedition test)
• migrate result of emlptest to mobile java platform that can run on the Java WTK emulator which emulates slow speed and network connection of mobile/portable devices
• problem: reduced API on Java ME: no full floating point operations support, had to look for replacement for several Math functions that are required (Math.exp, Math.random, ...)
• after research on the internet: “microfloat package” [11] was found and integrated, provides a replacement for missing floating-point operations by long/double conversion
Prove serialisation and de-serialisation of the emlp (ME)
• the working emlp (and emlpME for the mobile device) network structures were serialised into a file and de-serialised again (based on algorithms as described in [196]), i.e. the transfer of the knowledge base over a dataline was tested (proof of concept) and could basically be proved as the serialised and de-serialised network produced exactly the same results when classifying a dataset sequence.
Consideration of further steps
• as the development of a simple working ECoS system from scratch
succeeded (emlp and emlpME) and also the serialisation and de-serialisation could be proved as well, the former steps can be seen as “proof of
concept milestone (1)”, because it proves the development of an ecos that can be "moved" via state serialisation.
• after this milestone had been achieved, a short consideration of the required next steps was performed:
G the next step is to build an unbalanced system simulation, with the classifier moving from one side to the other, which represents the second proof-of-concept milestone (2).
Completion of the simulation environment
• a complete simulation environment is required in which the above systems can be tested “fully integrated” and where the defined numbers (as listed in section 6.2.5) can be measured
• the components from milestone 1 were taken and integrated into a fully working simulation environment to start the experiments:
G two nodes to be simulated are required: mobile and back-end/server were implemented
G mobile was run as emulated mobile device within the java wireless toolkit (Java WTK) as the WTK is able to emulate the slow processor speed and network bandwidth of mobile devices G the server was run as standard java application on a PC
G the data generator produced several different scenarios for the simulation runs, each scenario containing about 1000 records
saved into an XLS file to reproduce simulation runs with identical datasets (required to compare MECoS to traditional approaches)
6.4
Summary
Based on the author’s previous knowledge, the Java-Platform [6] was
chosen as the development environment for the experiments. The Java-Platform also delivers a practical tool, the Java Wireless Toolkit [7], which is used
for mobile application development and is capable of simulating the limited resources of mobile devices. The back-end modules have been implemented as a standalone Java application, communication between mobile and back-end nodes is performed via standard TCP/IP sockets.
As there is no implementation of an ECoS-based system publicly available, an ECoS implementation based on the concepts as described in [110] had to be built (as suggested after Prof. Kasabov had been contacted directly). The self-implemented ECoS was built upon a freely available Java implementation of a standard multilayer perceptron. As initial tests of the self-implemented ECoS-paradigm-based online machine learning algorithm were promising, a complete simulation environment for the conduction of experiments was built upon this initial prototype: the MECoS system. During the development of the simulation environment it was considered right from the start, that the classifier had to run in different classification modes. The simulation environment has to be able to run different algorithm types for comparison: the presented MECoS approach and a traditional, static client-server based approach (CS). Every simulation
run was performed under identical conditions in MECoS and also CS mode to compare the performance of those two approaches by the conducted measurements.
Conducting experiments with physiological data from real patients is affected by ethical restrictions and brings up a lot of bureaucracy. It is also very time-consuming and expensive and in some cases even impossible to do. As the specific use case of mobile care is not really relevant for the MECoS classifier, the experiments were conducted with artificially generated patient data but with the aim of being as realistic as possible.
Although, of course, the generated data cannot be compared to real physiological input data measured by sensors attached to a human being, the MECoS
approach learns the individual characteristics of this generated data - similar to the fact that every human patient is individually different and “delivers” individually different physiological measurements to the algorithm that has to learn it and adapt itself to it.
A data generator tool was developed within this research project to generate masses of physiological input data time series in different “scenarios”. The datasets consist of realistic value ranges of e.g. blood sugar, body temperature, blood oxygen level etc. The scenarios can be chosen for each time series that is being generated and saved as a database. A scenario defines the characteristic of the curves of the input data measures, e.g. normal, active/sport, emergency.
Based on various generated databases, the experiments could be conducted and compared to identical input data sequences as often as required: compare different algorithm types results, refine the algorithms, etc. To
be able to measure and compare the outcome of the experiments, some key numbers have been defined which represent the resource intensity or economic mode of operation of the algorithm.
Results
7.1
Introduction
T
HE aim of this research is to provide an efficient distributed onlinemachine learning algorithm that is applicable within heterogeneous distributed systems and performs better than traditional (client-server based) approaches in terms of resource and energy/battery consumption. Experiments have been conducted running the prototype of the presented MECoS approach and for comparison with a classical client-server (CS) based approach. The experiments have been performed within absolutely identical environments (same hardware, same input data).
From the numerous measurements collected during the experiments, a set of important key numbers was selected and analysed to derive metrics and compare the two different algorithms.
The results show significant improvement with the MECoS approach, it results in an average reduction of Processor Cycles used by about 30%, and an 12% improvement in required communication efforts.
A visual presentation of outcome and evaluation is provided by the following histograms: The first two compare the distribution of the overall number of processor cycles used (see figure 7.1). The shift of the bars to reduced processor cycles usage of the MECoS approach compared to the CS approach can be observed clearly:
Figure 7.1: Histograms of Processor Cycles Used - CS compared to MECoS
The second major issue related to efficiency and resource consumption are the required communication efforts (sending and receiving data is very expensive for mobile devices). The following two histograms visualise the distribution of required communication efforts of the MECoS approach compared to the traditional Client Server approach (see figure 7.2). The
distribution, which mainly depends on the characteristics of the generated input data, is not as smooth as with processor cycles. But the shift of the bars to significantly reduced overall communication efforts (C = n / S, where n is the number of connections that were opened and S is the amount of data that was transferred) of the MECoS approach compared to the Client Server (CS) approach can be observed, too:
Figure 7.2: Histograms of Communication Efforts - CS compared to MECoS
The following sections present the results of the experiments more closely. The values investigated and the numbers used to compare the approaches are derived, the measures of the important numbers are explained and evaluated in more detail.