A portable EEG signal acquisition system and a limited-electrode channel classification network for SSVEP

An Y., Wong J., Ling S.H.

Electroencephalogram (EEG)-based brain-computer interfaces (BCI) have been considered a prevailing non-invasive method for collecting human biomedical signals by attaching electrodes to the scalp. However, it is difficult to detect and use these signals to control an online BCI robot in a real environment owing to environmental noise. In this study, a novel state recognition model is proposed to determine and improve EEG signal states. First, a Long Short-Term Memory Convolutional Neural Network (LSTM-CNN) was designed to extract EEG features along the time sequence. During this process, errors caused by the randomness of the mind or external environmental factors may be generated. Thus, an actor-critic based decision-making model was proposed to correct these errors. The model consists of two networks that can be used to predict the final signal state based on both the current signal state probability and past signal state probabilities. Subsequently, a hybrid BCI real-time control system application is proposed to control a BCI robot. The Unicorn Hybrid Black EEG device was used to acquire brain signals. A data transmission system was constructed using OpenViBE to transfer data. An EEG classification system was built to classify the BCI commands. In this experiment, EEG data from five subjects were collected to train and test the performance and reliability of the proposed control system. The system records the time spent by the robot and the moving distance. Experimental results were provided to demonstrate the feasibility of the real-time control system. Compared to similar BCI studies, the proposed hybrid BCI real-time control system can accurately classify seven BCI commands in a more reliable and precise manner. Overall, the offline testing accuracy was 87.20%. When we apply the proposed system to control a BCI robot in a real environment, the average online control accuracy is 93.12%, and the mean information transmission rate is 67.07 bits/min, which is better than those of some state-of-the-art control systems. This shows that the proposed hybrid BCI real-time control system demonstrated higher reliability, which can be used in practical BCI control applications. © 2024 Elsevier Inc. All rights reserved.

Li X., Yang S., Fei N., Wang J., Huang W., Hu Y.

The application of wearable electroencephalogram (EEG) devices is growing in brain–computer interfaces (BCI) owing to their good wearability and portability. Compared with conventional devices, wearable devices typically support fewer EEG channels. Devices with few-channel EEGs have been proven to be available for steady-state visual evoked potential (SSVEP)-based BCI. However, fewer-channel EEGs can cause the BCI performance to decrease. To address this issue, an attention-based complex spectrum–convolutional neural network (atten-CCNN) is proposed in this study, which combines a CNN with a squeeze-and-excitation block and uses the spectrum of the EEG signal as the input. The proposed model was assessed on a wearable 40-class dataset and a public 12-class dataset under subject-independent and subject-dependent conditions. The results show that whether using a three-channel EEG or single-channel EEG for SSVEP identification, atten-CCNN outperformed the baseline models, indicating that the new model can effectively enhance the performance of SSVEP-BCI with few-channel EEGs. Therefore, this SSVEP identification algorithm based on a few-channel EEG is particularly suitable for use with wearable EEG devices.

Zhang S., An D., Liu J., Chen J., Wei Y., Sun F.

The SSVEP-based paradigm serves as a prevalent approach in the realm of brain-computer interface (BCI). However, the processing of multi-channel electroencephalogram (EEG) data introduces challenges due to its non-Euclidean characteristic, necessitating methodologies that account for inter-channel topological relations. In this paper, we introduce the Dynamic Decomposition Graph Convolutional Neural Network (DDGCNN) designed for the classification of SSVEP EEG signals. Our approach incorporates layerwise dynamic graphs to address the oversmoothing issue in Graph Convolutional Networks (GCNs), employing a dense connection mechanism to mitigate the gradient vanishing problem. Furthermore, we enhance the traditional linear transformation inherent in GCNs with graph dynamic fusion, thereby elevating feature extraction and adaptive aggregation capabilities. Our experimental results demonstrate the effectiveness of proposed approach in learning and extracting features from EEG topological structure. The results shown that DDGCNN outperforms other state-of-the-art (SOTA) algorithms reported on two datasets (Dataset 1: 54 subjects, 4 targets, 2 sessions; Dataset 2: 35 subjects, 40 targets). Additionally, we showcase the implementation of DDGCNN in the context of synchronized BCI robotic fish control. This work represents a significant advancement in the field of EEG signal processing for SSVEP-based BCIs. Our proposed method processes SSVEP time domain signals directly as an end-to-end system, making it easy to deploy. The code is available at https://github.com/zshubin/DDGCNN.

Tao W., Wang Z., Wong C.M., Jia Z., Li C., Chen X., Philip Chen C.L., Wan F.

Karas K., Pozzi L., Pedrocchi A., Braghin F., Roveda L.

AbstractHuman-robot interaction is a rapidly developing field and robots have been taking more active roles in our daily lives. Patient care is one of the fields in which robots are becoming more present, especially for people with disabilities. People with neurodegenerative disorders might not consciously or voluntarily produce movements other than those involving the eyes or eyelids. In this context, Brain-Computer Interface (BCI) systems present an alternative way to communicate or interact with the external world. In order to improve the lives of people with disabilities, this paper presents a novel BCI to control an assistive robot with user’s eye artifacts. In this study, eye artifacts that contaminate the electroencephalogram (EEG) signals are considered a valuable source of information thanks to their high signal-to-noise ratio and intentional generation. The proposed methodology detects eye artifacts from EEG signals through characteristic shapes that occur during the events. The lateral movements are distinguished by their ordered peak and valley formation and the opposite phase of the signals measured at F7 and F8 channels. This work, as far as the authors’ knowledge, is the first method that used this behavior to detect lateral eye movements. For the blinks detection, a double-thresholding method is proposed by the authors to catch both weak blinks as well as regular ones, differentiating itself from the other algorithms in the literature that normally use only one threshold. Real-time detected events with their virtual time stamps are fed into a second algorithm, to further distinguish between double and quadruple blinks from single blinks occurrence frequency. After testing the algorithm offline and in realtime, the algorithm is implemented on the device. The created BCI was used to control an assistive robot through a graphical user interface. The validation experiments including 5 participants prove that the developed BCI is able to control the robot.

He C., Chen Y., Phang C., Stevenson C., Chen I., Jung T., Ko L.

Chen C., Ji Z., Sun Y., Bezerianos A., Thakor N., Wang H.

Chen J., Zhang Y., Pan Y., Xu P., Guan C.

Steady-state visual evoked potential (SSVEP) is one of the most commonly used control signals in the brain-computer interface (BCI) systems. However, the conventional spatial filtering methods for SSVEP classification highly depend on the subject-specific calibration data. The need for the methods that can alleviate the demand for the calibration data becomes urgent. In recent years, developing the methods that can work in inter-subject scenario has become a promising new direction. As a popular deep learning model nowadays, Transformer has been used in EEG signal classification tasks owing to its excellent performance. Therefore, in this study, we proposed a deep learning model for SSVEP classification based on Transformer architecture in inter-subject scenario, termed as SSVEPformer, which was the first application of Transformer on the SSVEP classification. Inspired by previous studies, we adopted the complex spectrum features of SSVEP data as the model input, which could enable the model to simultaneously explore the spectral and spatial information for classification. Furthermore, to fully utilize the harmonic information, an extended SSVEPformer based on the filter bank technology (FB-SSVEPformer) was proposed to improve the classification performance. Experiments were conducted using two open datasets (Dataset 1: 10 subjects, 12 targets; Dataset 2: 35 subjects, 40 targets). The experimental results show that the proposed models could achieve better results in terms of classification accuracy and information transfer rate than other baseline methods. The proposed models validate the feasibility of deep learning models based on Transformer architecture for SSVEP data classification, and could serve as potential models to alleviate the calibration procedure in the practical application of SSVEP-based BCI systems.

Li J., Wang F., Huang H., Qi F., Pan J.

The brain-computer interface (BCI) provides a direct communication pathway between the human brain and external devices. However, the models trained for existing subjects perform poorly on new subjects, which is termed the subject calibration problem. In this paper, we propose a semi-supervised meta learning (SSML) method for subject-transfer calibration. The proposed SSML learns a model-agnostic meta learner with existing subjects and then fine-tunes the meta learner in a semi-supervised learning manner, i.e. using a few labelled samples and many unlabelled samples of the target subject for calibration. It is significant for BCI applications in which labelled data are scarce or expensive while unlabelled data are readily available. Three different BCI paradigms are tested: event-related potential detection, emotion recognition and sleep staging. The SSML achieved classification accuracies of 0.95, 0.89 and 0.83 in the benchmark datasets of three paradigms. The runtime complexity of SSML grows linearly as the number of samples of target subject increases so that is possible to apply it in real-time systems. This study is the first attempt to apply semi-supervised model-agnostic meta learning methodology for subject calibration. The experimental results demonstrated the effectiveness and potential of the SSML method for subject-transfer BCI applications.

Zhang A., Li Q., Li Z., Li J.

Liu K., Yang M., Yu Z., Wang G., Wu W.

Object: Motor imagery (MI) is a mental process widely utilized as the experimental paradigm for brain-computer interfaces (BCIs) across a broad range of basic science and clinical studies. However, decoding intentions from MI remains challenging due to the inherent complexity of brain patterns relative to the small sample size available for machine learning. Approach: This paper proposes an end-to-end Filter-Bank Multiscale Convolutional Neural Network (FBMSNet) for MI classification. A filter bank is first employed to derive a multiview spectral representation of the EEG data. Mixed depthwise convolution is then applied to extract temporal features at multiple scales, followed by spatial filtering to mitigate volume conduction. Finally, with the joint supervision of cross-entropy and center loss, FBMSNet obtains features that maximize interclass dispersion and intraclass compactness. Main results: We compare FBMSNet with several state-of-the-art EEG decoding methods on two MI datasets: the BCI Competition IV 2a dataset and the OpenBMI dataset. FBMSNet significantly outperforms the benchmark methods by achieving 79.17% and 70.05% for four-class and two-class hold-out classification accuracy, respectively. Significance: These results demonstrate the efficacy of FBMSNet in improving EEG decoding performance toward more robust BCI applications. The FBMSNet source code is available at https://github.com/Want2Vanish/FBMSNet.

Yin H., Ji Z., Lian Z., Yang Y., Liu N., Wang H.

Pan Y., Chen J., Zhang Y., Zhang Y.

Abstract Objective. Steady-state visual evoked potentials (SSVEPs) based brain–computer interface (BCI) has received great interests owing to the high information transfer rate and available large number of targets. However, the performance of frequency recognition methods heavily depends on the amount of the calibration data for intra-subject classification. Some research adopted the deep learning (DL) algorithm to conduct the inter-subject classification, which could reduce the calculation procedure, but the performance still has large room to improve compared with the intra-subject classification. Approach. To address these issues, we proposed an efficient SSVEP DL NETwork (termed SSVEPNET) based on one-dimensional convolution and long short-term memory (LSTM) module. To enhance the performance of SSVEPNET, we adopted the spectral normalization and label smoothing technologies during implementing the network architecture. We evaluated the SSVEPNET and compared it with other methods for the intra- and inter-subject classification under different conditions, i.e. two datasets, two time-window lengths (1 s and 0.5 s), three sizes of training data. Main results. Under all the experimental settings, the proposed SSVEPNET achieved the highest average accuracy for the intra- and inter-subject classification on the two SSVEP datasets, when compared with other traditional and DL baseline methods. Significance. The extensive experimental results demonstrate that the proposed DL model holds promise to enhance frequency recognition performance in SSVEP-based BCIs. Besides, the mixed network structures with convolutional neural network and LSTM, and the spectral normalization and label smoothing could be useful optimization strategies to design efficient models for electroencephalography data.

Wu D., Jiang X., Peng R.

A brain-computer interface (BCI) enables a user to communicate directly with an external device, e.g., a computer, using brain signals. It can be used to research, map, assist, augment, or repair human cognitive or sensory-motor functions. A closed-loop BCI system performs signal acquisition, temporal filtering, spatial filtering, feature engineering and classification, before sending out the control signal to an external device. Transfer learning (TL) has been widely used in motor imagery (MI) based BCIs to reduce the calibration effort for a new subject, greatly increasing their utility. This tutorial describes how TL can be considered in as many components of a BCI system as possible, and introduces a complete TL pipeline for MI-based BCIs. Examples on two MI datasets demonstrated the advantages of considering TL in multiple components of MI-based BCIs. Especially, integrating data alignment and sophisticated TL approaches can significantly improve the classification performance, and hence greatly reduces the calibration effort.

Tonin L., Beraldo G., Tortora S., Menegatti E.

The growing interest in neurorobotics has led to a proliferation of heterogeneous neurophysiological-based applications controlling a variety of robotic devices. Although recent years have seen great advances in this technology, the integration between human neural interfaces and robotics is still limited, making evident the necessity of creating a standardized research framework bridging the gap between neuroscience and robotics. This perspective paper presents Robot Operating System (ROS)-Neuro, an open-source framework for neurorobotic applications based on ROS. ROS-Neuro aims to facilitate the software distribution, the repeatability of the experimental results, and support the birth of a new community focused on neuro-driven robotics. In addition, the exploitation of Robot Operating System (ROS) infrastructure guarantees stability, reliability, and robustness, which represent fundamental aspects to enhance the translational impact of this technology. We suggest that ROS-Neuro might be the future development platform for the flourishing of a new generation of neurorobots to promote the rehabilitation, the inclusion, and the independence of people with disabilities in their everyday life.

Vaida C., Rus G., Pisla D.

Neurological diseases leading to motor deficits constitute significant challenges to healthcare systems. Despite technological advancements in data acquisition, sensor development, data processing, and virtual reality (VR), a suitable framework for patient-centered neuromotor robot-assisted rehabilitation using collective sensor information does not exist. An extensive literature review was achieved based on 124 scientific publications regarding different types of sensors and the usage of the bio-signals they measure for neuromotor robot-assisted rehabilitation. A comprehensive classification of sensors was proposed, distinguishing between specific and non-specific parameters. The classification criteria address essential factors such as the type of sensors, the data they measure, their usability, ergonomics, and their overall impact on personalized treatment. In addition, a framework designed to collect and utilize relevant data for the optimal rehabilitation process efficiently is proposed. The proposed classifications aim to identify a set of key variables that can be used as a building block for a dynamic framework tailored for personalized treatments, thereby enhancing the effectiveness of patient-centered procedures in rehabilitation.