Deep Learning Algorithms and Brain-Computer Interfaces

As part of a research team, I wanted to explain how Deep learning (DL) has lifted the performance of brain-computer interface systems (BCI) significantly in recent years.

For those not familiar with Brain-Computer Interface, BCI is a system that translates activity patterns of the human brain into messages or commands to communicate with other devices.

As you can imagine, designing a BCI is a complex task that requires multi-disciplinary knowledge in computer science, engineering, signal processing, neuroscience, etc.

To use a BCI, two phases are generally required:

Calibration is challenging in BCI, because the signal-to-noise ratio (SNR) is unfavorable and the subject-to-subject variability is large. Depending on the type of paradigm chosen, the required time for calibration can differ. Even though, the calibration time can be partly reduced.

The key challenge of BCI is to recognize human intents accurately given the meager Signal-to-Noise Ratio (SNR) of brain signals. The truth is both low classification accuracy and poor generalization ability limit the real-world application of BCI.

To overcome the above challenges, deep learning techniques have been used to deal with the brain information in the past few years. Differing from traditional machine learning algorithms, deep learning can learn specific high-level features from brain signals without manual feature selection, and its accuracy scales well with the size of the training set. Moreover, deep learning models have been applied to several types of BCI signals (e.g., spontaneous EEG, ERP, fMRI).

Why Deep Learning?

First, brain signals are easily corrupted by various biological (e.g., eye blinks, muscle artifacts, fatigue and concentration level) and environmental artifacts (e.g., environmental noise).

There are many difficulties in working with EEG.Since the main task of BCI is brain signal recognition, the discriminative deep learning models are the most popular and powerful algorithms.

A BCI may monitor brain activity via a variety of methods, which can be coarsely classified as invasive and non - invasive . Most non - invasive BCI systems use electroencephalogram (EEG) signals; i.e., the electrical brain activity recorded from electrodes placed on the scalp.

It is difficult to make sense of brain-activity that propagates from the neurons speaking to each other, through the skull, through one’s scalp, and just barely into the EEG sensor. Generally, EEG data is very noisy in the sense that it is very hard to get a clear signal for something specific.

Therefore, it is crucial to distill informative data from corrupted brain signals and build a robust BCI system that works in different situations.

Moreover, BCI has a low SNR due to the non-stationary nature of electrophysiological brain signals.

The accuracy in classifying electroencephalographic (EEG) data in brain-computer interfaces (BCI) depends on the number of measuring channels, the amount of data used to train the classifier, and the signal-to-noise ratio (SNR). Of all those factors, the SNR is the hardest to adjust in real-life applications.

Although several preprocessing and feature engineering methods have been developed to decrease the noise level, such methods (e.g., feature selection and extraction both in the time domain and frequency domain) are time-consuming and may cause information loss in the extracted features.

Third, feature engineering highly depends on human expertise in the specific domain. Human experience may help capture features on some particular aspects but prove insufficient in more general conditions. Therefore, an algorithm is required to extract representative features automatically.

Deep learning provides a better option to automatically extract the distinguishable features.

Moreover, a majority of current machine learning research focuses on static data and therefore cannot classify rapidly changing brain signals accurately. It generally requires novel learning methods to deal with dynamical data streams in BCI systems.

Until now, deep learning has been applied extensively in BCI applications and shown success in addressing the above challenges.

Deep learning has three advantages. First, it avoids the time-consuming preprocessing and feature engineering steps by working directly on raw brain signals to learn distinguishable information through back-propagation. Furthermore, deep neural networks can capture both representative high-level features and latent dependencies through deep structures.

Finally, Deep learning algorithms are shown to be more powerful than traditional classifiers such as Support Vector Machine (SVM) and Linear discriminant analysis (LDA). It makes sense because almost all the BCI issues can be regarded as a classification problem.

DL Algorithms used in BCI

CNN is the most popular DL model in BCI research, which can be used to exploit the latent spatial dependencies among the input brain signals like fMRI image, spontaneous EEG, and so on.

CNN has achieved great success in some research areas which makes it extremely “scalable” and feasible (through the available public code). Thus, the BCI researchers have more chance to understand and apply CNN on their works.

Generative deep learning models are mostly used to generate training samples or data augmentation. In other words, generative deep learning models play a supporting role in BCI area to enhance the training data quality and quantity. In BCI scope, generative algorithms are mostly used in reconstruction or generate a batch of brain signals samples to enhance the training set. Generative models commonly used in BCI include variational Autoencoder (VAE),Generative Adversarial Networks (GANs), etc.

Deep Belief Networks are also used in BCI for feature extraction. Even though a growing number of publications focus on adopting CNN or hybrid models for both feature learning and classification.

RNN and CNN are illustrated having excellent temporal and spatial feature extraction ability, it’s natural to combine them for both temporal and spatial feature learning.

Future challenges

One promising research area for deep learning-based BCI is the development of a general framework that can handle various BCI signals regardless of the number of channels used for signal collection, the sample dimensions, and stimulation types (visual or audio stimuli), etc.

The general framework would require two key capabilities:

• Attention mechanism
• Ability to capture latent feature.

The former guarantees the framework can focus on the most valuable parts of input signals and the latter enables the framework to capture the distinctive and informative features.

Until now, most BCI classification tasks focus on person-dependent scenarios, where the training and the testing sets come from the same person. The future direction is to realize person-independent classification so that the testing data will never appear in the training set. High-performance person-independent classification is compulsory for the wide application of BCI Systems in the real-world.

One possible solution to achieving this goal is to build a personalized model with transfer learning.

Future of BCI

When it comes to more advanced ideas, we may still be years away due to the more complex functions of the brain that aren’t as easy to understand. We’re still learning how the brain can create these complex functions, and while some have tried even very rudimentary attempts in non-human species, the results are not up to our expectations.

Based on some research, there are currently less than 100 people using some of the earlier forms of this technology.

In terms of methodology, complex network theory is now in its early years... It needs to reach maturity before we will see the impact on our understanding of the intrinsic mechanisms of complex networks such as the brain. Current technology can only give us access to certain parts of the nervous system.

In Computer Science, many things remain to be done on decoding neural data in a time-efficient manner with more efficient algorithms and data structures.

For more information, I recommend you to read this excellent study that helped me a lot writing this article:

- “ A Survey on Deep Learning based Brain Computer Interface:Recent Advances and New Frontiers ”