Comparing machine learning approaches for motor-activity-related brain computer interfaces

• ¹ Drexel University, School of Biomedical Engineering, Science & Health Systems, United States
• ² Drexel University, Cognitive Neuroengineering and Quantitative Experimental Research (CONQUER) Collaborative, United States
• ³ University of Pennsylvania, Department of Family and Community Health, United States
• ⁴ Children's Hospital of Philadelphia, The Division of General Pediatrics, United States

INTRODUCTION: A brain-computer interface (BCI) is a system that detects consistent spatiotemporal patterns in human brain activity that are related to select mental tasks, such as performing motor imagery, or cognitive workload (Bashashati, Ward, Birch, & Bashashati, 2015; Wolpaw, Birbaumer, McFarland, Pfurtscheller, & Vaughan, 2002). One of the main goal of an active BCI is to provide a new channel of output for the brain that requires voluntary adaptive control by the user, mainly used as a neurorehabilitation tool to improve motor or cognitive performance for people with motor disorders, such as spinal cord injury, amyotrophic lateral sclerosis (ALS), or people in the persistent locked-in state (LIS) (Coyle, Ward, Markham, & McDarby, 2004; Naseer & Hong, 2015; Vallabhaneni, Wang, & He, 2005). Among the various means to measure the brain activity, the electroencephalography (EEG)-based BCI systems can measure the changes in the brain activity over short periods of time, non-invasive and versatile, but spatial localization and the signal to noise ratio can be low (Bashashati et al., 2015). Functional near infrared spectroscopy (fNIRS) as an emerging optical neuroimaging technique, is relatively new in BCI. fNIRS is not susceptible to electrical noise or muscle activity, and also provides a balanced trade-off between temporal and spatial resolution. Even though the use of fNIRS-BCI has been only a very recent start, there’s accelerated progress of fNIRS-BCI research that shows potential as a supplement for electroencephalography (EEG) BCI (Naseer & Hong, 2015). Motor-activity-related mental tasks are widely adopted for BCIs as they are a natural extension of movement intention. Motor imagery (MI) is one of the remarkable abilities of the mind that suitable for use in BCI system, as it refers to the mental representation of an overt action without any concomitant motor execution (ME) (Batula, Mark, Kim, & Ayaz, 2017; Jeannerod, 1994; Xu et al., 2014). The aim of this study is to systematically compare performance of machine learning approaches for developing optimal fNIRS and EEG based motor BCI systems. METHODS: The performance of the following machine learning algorithms were compared using fNIRS and EEG data: Naïve Bayes (NB), Linear Discriminant Analysis (LDA), Logistic Regression (LR), Support Vector Machines (SVM) and Multi-layer Perception (MLP). The fNIRS ME/MI based BCI, and EEG ME/MI BCI datasets were from earlier studies (Batula, Kim, & Ayaz, 2017; Cho, Ahn, Ahn, Kwon, & Jun, 2017; Tangermann et al., 2012). For each, same set of features were extracted from the preprocessed fNIRS and EEG datasets. NB classifiers are a collection of classification algorithms based on Bayes’ Theorem and particularly suited when the dimensionality of the feature space is high, NB classifiers can often outperform more sophisticated methods. LDA is the most commonly used classification in fNIRS-BCI studies, because of its simplicity and low computational requirements, it is highly suitable for online BCI systems. LR as a discriminative learning classifier that directly estimated the parameters of the posterior distribution function, it is intrinsically simple, but is prone to overfitting when data space is sparse and of high dimension. SVM with the regularization parameter to reduce errors on training set and kernel function to define non-linear decision boundaries, non-linear SVM provides more flexible decision boundary that can result in an increased classification accuracy. MLP classifier has the flexibility and capability of approximating any function with enough number of neurons in the hidden layer, but can be easily overfitting. In a preliminary analysis, classification using all the algorithms was performed on four-class MI-based fNIRS BCI datasets. After FIR filter with 0.1Hz cutoff frequency, an automatic data-quality analysis was used to determine which optodes and trials should be removed due to poor data quality, then average HbO level for the first two seconds of the task was subtracted from 15s task duration right after preprocessing. Eleven 5s segmented data of HbO were regenerated afterwards, and signal slope (SS) and signal mean (SM) were computed, resulted in 528 features in feature space. RESULTS: The preliminary results on MI-based fNIRS BCI dataset showed, LR classifier achieved relative higher average classification accuracy than other classifiers: LR: 31.02%, SVM: 30.57%, MLP: 30.27%, NB: 30.10%, LDA:28.45%. The confusion matrix of LR classification results for Subject 1 shows a relative strong diagonal pattern, as expected for a well-performing classifier. Interestingly, left hand and right hand are almost never misclassified as the opposite hand, left hand and left foot task was also seldom misclassified as the other side, LDA and SVM classification also showed pattern similar to LR. While NB classifier showed relative focus on correct classification on left side tasks compared to right side tasks. For subject 6, LR classifier showed stronger right-side tasks accuracy than left side tasks, while NB classifier displayed stronger left side tasks accuracy than right side tasks. LR, LDA and SVM classifier all showed a focus on correct classification on hand tasks compared to foot tasks, as shown in Fig 1. These results indicated: i) LR could achieve relative higher average classification accuracy compared with other four classifiers; ii) for some subjects, the classification pattern of LR, LDA and SVM classifiers are similar, which implies that LR could be a potential substitute of LDA and SVM for BCI studies. CONCLUSION: These results indicated LR classifier slightly outperformed other classifiers, unlike most fNIRS BCI studies, which selected LDA or SVM as the best classifier, and LR could be served as a potential replacement of LDA or SVM classifier in BCI studies. Further evaluation includes comparison of different machine learning algorithms on ME-based fNIRS BCI and EEG BCI datasets with relative larger sample size. Fig 1. Confusion matrices for the two subjects showing different pattern between LR, LDA, SVM vs NB classification results.