Decoding attention at higher levels of linguistic processing using EEG

Recent discoveries have shown that single trial (~60s) unaveraged EEG data can be decoded to determine attentional selection in a multi-speaker environment. This is achieved by using recorded neural data to reconstruct an estimate of the speech envelope and then comparing that reconstruction with the attended and unattended speech streams. This result is particularly impressive given that it is based on the envelope of speech. There are two reasons for this: 1) the envelope is a very simplified representation of speech meaning that the corresponding EEG measures may be relatively poor in terms of reflecting speech-related activity; and 2) attention effects tend to be stronger at higher levels of sensory-perceptual hierarchies meaning that attention measures based on the envelope may be quite small. Here, we sought to investigate whether the addition of EEG measures reflecting higher-level linguistic processing could lead to improved decoding of attention in a “cocktail party” experiment.

We recorded EEG as subjects listened to one of two concurrently presented stories. We represented each story in terms of its semantic content by using computational language models to quantify the meaning of each word in a sentence in terms of how semantically dissimilar it was to its preceding context. We then regressed the EEG data against this semantic representation to produce Temporal Response Functions (TRFs; i.e., beta regression weights) for both the attended and unattended story.

These TRFs display a prominent negativity at time-lags of ~200-600ms over centro-parietal electrodes, sharing similar characteristics to those of the classic N400 response. Importantly, this negativity is consistent across subjects and is exquisitely sensitive to whether or not subjects were understanding the speech they heard. As such, it is consistently present across subjects for attended speech and absent for unattended speech. By including measures of this semantic TRF along with envelope reconstruction, we show improved decoding of cocktail party attention using unaveraged EEG.