Analysis of the individual listening effort reflected by the pupillary responses during speech perception in noise

Pupillometry has been applied to investigate listening effort involved in speech recognition in adverse listening situations. It has been demonstrated that the pupil dilation can be sensitive to the cognitive resources that are allocated to perform a task such as speech recognition in noise. Commonly, pupillary response is analyzed based on the measure of the maximum diameter or the mean diameter of the dilating pupil while performing a task. Recent studies performed a Growth Curve Analysis (GCA, multilevel regression technique designed for analysis of time course or longitudinal data) on the pupillary data in order to account for time-dependent changes of the pupillary response. So far, analysis of the pupillary responses aimed to examine changes in listening effort at a group level. The objective of the current study was to analyze and classify individual pupil traces in order to examine individual’s listening effort. The present work provides some preliminary results on the classification of the high/low listening effort as reflected by the individual pupil traces recorded during a speech-in-noise test. The approach was based on defining a multilevel regression to model pupil responses. Furthermore machine learning (ML) algorithms were applied to derive a deeper knowledge on changes in pupil response due to changes in the listening effort. In a first step, GCA, with a polynomial functional basis, was applied to fit the pupil traces from a limited set of 22 individuals performing a task with either high or low demands (recorded by Wendt et al., 2017). By employing the GCA, time-dependent, differentiable features of the individual’s pupil responses were extracted. In a second step, it was studied how these estimated GCA parameters can be used to build a ‘high/low listening effort’ classifier by assessing the performances of different classifiers found in ML research. Preliminarily finings indicated that the best obtained detection accuracy of listening effort was higher than 80% (36/44). Our preliminary results demonstrated that linear and cubic parameters were highly relevant for separating the pupil curves. Moreover, both sensitivity (correctly identified high effort) and specificity (correctly identified low effort) of the classifier were above 80%. Analysis of the individual pupil traces showed furthermore that some pupil responses were too weak or too irregular within the class to be modelled with the chosen 3rd order polynomial basis. In general, this study provides a first approach to quantify and classify individual’s pupillary response recorded within a speech-in-noise test. First results are encouraging that this approach can give an insight into the individual’s listening effort involved in speech recognition in adverse listening situations.