Projects

Decoding concepts from their EEG signature

We have developed an innovative method for determining what concept is being thought about from its EEG signature. This method leverages our groundbreaking work (published in Science in 2008, Mitchell et al.) that demonstrated the ability to decode concepts from their fMRI signature. EEG is two orders of magnitude more accessible and affordable than fMRI, enabling the widespread use of this method in many non-fMRI settings.

The critical insight that enabled this breakthrough was the idea of collecting EEG data concurrently with fMRI, and then using the fMRI data (which is known to allow concept decoding) to guide us to the relevant aspects of the EEG signal. With this guidance (of EEG feature selection and weighting), we accomplish fairly high levels of decoding accuracy. A participant can think of nose or apple or hammer and the machine learning program can identify the concept from its EEG signature with a high rank accuracy, approaching and sometimes exceeding fMRI levels of accuracy.

Furthermore, because the fMRI signatures are similar across people, once they are acquired from a small sample of fMRI participants, they can be applied to accurately guide the EEG decoding of any number of new EEG participants, without acquiring any fMRI data from them. The use of the method becomes free from any further fMRI acquisition.

The scientific innovation in this study is that we decode the content of the thought from its EEG signal and not just its temporal characteristics, as most EEG/ERP research does. The EEG signal can tell you what concept a person is thinking with reasonable accuracy, if you know where to look. This is a scientific advance linking electrical signals to high-level psychological states.

This technology can be of enormous value to people with a communication disorder, such as an expressive aphasia. They could simply think of a concept (within some vocabulary) while wearing an EEG cap and an automated system that could identify the concept and would say it aloud. A practically useful, non-invasive brain-computer interface has been a research target for decades, and now we have the underlying method in hand. This research was supported by NIMH.