Iberdrola S.A

The classification delay result was summarized and shown in Figure 6.3. On average the moving average method showed lower classification delay than the biased- classification method. According to the Kolmogorov-Smirnov goodness-of-fit hypothesis test, the distribution of delay time on the moving average method was not normally distributed (n=443, P= 1.07×10-107_{). Wilcoxon rank sum test showed the delay time of the}

two online classification method was significantly different (n1 = 443, n2 = 560, P =

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% False Positive Rate

Precision Accuracy F1 score Cohen's Kappa Score P er ce n t

Figure 6.3 Classification delay summary for moving average classification and biased classification

6.4. Discussion

In this chapter, two prediction filtering methods were proposed and evaluated with EEG data in a pseudo-online way. EEG data collected from six weeks of BCI

involved rehabilitation training from one case participant were used. In the first method, a threshold value was calculated depending on the training sample size, and a biased- classification method was then built utilizing the threshold value. In the second method, a moving average method was proposed to process the probability output of the

classifier, to filter out the “classification noise”. The two prediction filtering methods were also evaluated in this chapter with EEG data from one case participant with chronic stroke during six weeks of rehabilitation training.

Although the highest cross-validation accuracy was 80.1% for the model generation[192], which utilized the EEG data collected in the three baseline assessments, the overall average pseudo-online classification accuracy during six weeks of training barely surpassed the randomness level, with the EEG data collected in the six weeks of BCI involved rehabilitation training. The daily EEG acquisition setups

did introduce a lot of influence, which resulted in the variation between the training set for model generation and the actual rehabilitation training. However, the participant was able to learn how to use the training system very effectively. It only takes a few sessions before the participant learns how to use the training system. In the last week of the case study, the participant was able to control the BCI with very high accuracy (average 90.6%), using the moving average method proposed in this chapter [192].

Based on the evaluation results, both of the proposed prediction filtering methods outperformed the non-specific online classification method in all evaluations except F1

score. However, the F1 score does not take the true negatives into account [205], and

Cohen’s kappa score was suggested to be a better measure [204]. The result on Cohen’s kappa score suggested that the moving average method had better performance. In addition, the moving average method had the lowest false positive accuracy, which is crucial in a binary online classification for applications like

rehabilitation. Lower false positive would decrease the possibility of triggering assistance by mistake and ensure the participants’ focus on the training protocol.

Since the proposed biased classification used the same number of data points with the non-specific online classification method, the delays caused by the BCI algorithm should be intrinsically the same with these two methods. Considering the moving average method requires 8 recent predictions to calculate the mean, the moving average method should require more time to respond. This initial hypothesis has been supported by Figure 6.3, where the response delay for biased-classification method is significantly shorter than the moving average method.

The relatively low accuracy of online classification is still an issue for the application of BCIs. The two prediction filtering methods proposed in this study are still simple and naïve. More sophisticated algorithms should be developed to minimize the variations between the training set for model generation and the actual online

application. Also, the result of this study has been limited by the population of the participants. The possibility of generalizing the result and conclusion of the proposed methods with a large population is still unknown. A future study with more participants should be conducted to investigate this potential problem.

6.5. Chapter Summary

The goal set by Objective 2 was addressed in this chapter, by investigating the two prediction filtering methods in an online BCI application. The scope of this chapter was to introduce and evaluate two prediction filtering methods for long-term BCI

applications like rehabilitation. Two naïve prediction filtering methods were first proposed in this chapter. In the biased classification method, a threshold value was calculated depending on the training sample size, and a biased-classification method was then built utilizing the calculated threshold value. A moving-average method was proposed to process the probability output of the classifier, to filter out the “classification noise” and calibrate the online classification process with respect to the variation introduced by the EEG acquisition set-up. The performance of two prediction filtering methods was also evaluated in this chapter with EEG data from one case participant with chronic stroke during six weeks of rehabilitation training. Both of the proposed two prediction filtering methods outperformed the method of directly applying the BCI model on the buffered EEG data. Between the two proposed methods, the moving-average method achieved significantly higher classification accuracy than the biased classification method, yet the biased classification method showed a significantly faster response than the moving- average method.

6.6. Contributions, limitations and future work

This chapter evaluated two methods of improving the BCI application

performance by filtering the classification output of the EEG model. Although the offline EEG analysis methods have been extensively researched, the performance of the online BCI application is still not satisfactory, especially for users with chronic stroke, as

discussed in Chapter 3. Therefore, the process of configuring the BCI application using the EEG model generated from offline analysis requires further investigation and improvement.

The two methods of filtering the classification results of the EEG model

presented in this study contribute to filling in the gap between the offline EEG analysis and practical BCI application. With the same offline analysis method, the performance of the BCI application can be improved. This study also benefits the healthcare community

and the users of the BCI system, as it contributes to the reliability of the current BCI technology, making it less demanding for rehabilitation applications.

Although the two proposed methods were investigated with EEG data collected from six weeks of BCI involved rehabilitation training, only one participant with chronic stroke was recruited. This limits the generalizability of the results obtained in this study. A future study involving more participants with chronic stroke is necessary to consolidate the results of the two proposed methods. In addition, the threshold value for the moving average method was calculated using 30 seconds of resting state EEG data, which were determined based on previous studies with healthy participants and participants with chronic stroke. A study investigating the impact of such hyperparameters in the future will help better understand the roles of hyperparameters in the methods proposed in this chapter, as well as develop new methods to improve the performance of the BCI.