Mind Reading: What linguistic concepts can we decode with fMRI?

We are at an especially opportune time in the history of the study of human cognition. Brain imaging technology allows us to directly observe brain activity associated with cognitive processes. Techniques from statistics and machine learning allow us to construct quantitative computational models that describe these cognitive brain processes. More importantly, they allow us to construct mental state decoders that accurately predict certain aspects of thought from measured brain activity. In addition to the scientific impact of better understanding the representation and processing of human cognition, this research will lead to many applications and broad impacts. For example, a brain-computer interface (BCI) device that could decode internal speech may enable locked-in patients to communicate. A device that could detect mental state could be used to detect goals, needs, and priorities of an operator.

In this talk, I will talk about some of the state-of-the-art mental state decoding research conducted at CMU. I will talk about how we build machine learning classifiers to decode the fMRI brain activity recorded when people contemplate about objects from a range of categories (e.g. tools, dwellings), or even multi-word expressions, such as adjective-noun phrases (e.g. small cup) and noun-noun concept combinations (e.g. tomato cup). Finally, I will talk about some of my recent effort to build mental state decoders based on the portable, commercially-available EEG devices to decode human emotions (happy, sad, funny, angry).