Perceiving non-speech as speech based on a moving mouth. Findings from infants, children, and adults

Martijn Baart

BCBL. Basque Center on Cognition, Brain and Language, Donostia–San Sebastián, Spain

Department of Cognitive Neuropsychology, Tilburg University, Tilburg, The Netherlands

When the natural richness of the auditory speech signal is reduced to a few sinusoids, most listeners perceive this so-called "sine-wave speech" (SWS) as non-speech bleeps or whistles. However, when listerens are made aware of the phonetic content of the stimulus, they cannot switch back to a perceptual non-speech mode and will continue to hear the sounds as speech. Here, I will show that when infants see the articulatory gestures of a speaker while hearing SWS, they can detect the correspondence between the sound and the articulating face. However, knowing the phonetic content of the SWS stimulus induces a boost in performance after about six and a half years of age, and yields speech-like behavioral and electrophysiological patterns of audiovisual integration in adults. I will argue that the temporal correlation between the time-varying properties of SWS and an articulating mouth can drive audiovisual integration up to some extent, but that phonetic knowledge about the SWS stimulus is needed to achieve full speech-like audiovisual integration.