Even proficient second language speakers need to pay attention to the speaker's mouth
In an adult conversation, speakers tend to look at the othersʼ eyes when talking in their native language. However, this does not happen when we try to understand a person talking in a second language (L2), for instance, if an English speaker is talking to someone in Spanish. In this case, according to a study of the UB which has been published in Language, Cognition & Neuroscience, we attend to the talkerʼs mouth to support understanding, and this occurs to low-level learners and proficient L2 speakers. The conclusion of this study is specially relevant now due to the general use of masks, and suggests this use could worsen the comprehension of a spoken language, not only for people with hearing problems -as said in the media- but also for those who live abroad or who need to communicate in a non-native language.
The study is signed by Joan Birulés, PhD student at the Department of Cognition, Development and Psychology of Education of the University of Barcelona, together with the researchers of the Faculty of Psychology of the same university Ferran Pons and Laura Bosch, in collaboration with David Lewkowicz, from Haskins Laboratories (United States).
In an adult conversation, speakers tend to look at the othersʼ eyes when talking in their native language. However, this does not happen when we try to understand a person talking in a second language (L2), for instance, if an English speaker is talking to someone in Spanish. In this case, according to a study of the UB which has been published in Language, Cognition & Neuroscience, we attend to the talkerʼs mouth to support understanding, and this occurs to low-level learners and proficient L2 speakers. The conclusion of this study is specially relevant now due to the general use of masks, and suggests this use could worsen the comprehension of a spoken language, not only for people with hearing problems -as said in the media- but also for those who live abroad or who need to communicate in a non-native language.
The study is signed by Joan Birulés, PhD student at the Department of Cognition, Development and Psychology of Education of the University of Barcelona, together with the researchers of the Faculty of Psychology of the same university Ferran Pons and Laura Bosch, in collaboration with David Lewkowicz, from Haskins Laboratories (United States).
Methodology of the study
In order to carry the study out, researchers recorded a trilingual girl (Catalan mother, English father) while she told 1-minute stories in Catalan, Spanish and English. With a visual monitoring device, they recorded the eye-gaze of university students from Barcelona and Boston while watching the videos, and then they checked the time of visual attention to the area of the mouth and eyes of the speaker. “We conducted the study in two linguistically different places to rule out a potential specific effect of the used languages (for the students in Barcelona, English is L2, and for those in Boston, Catalan and Spanish). We could then attribute the found differences listening to a native or a second language, instead of the features of the language itself”, notes Joan Birulés.
In the first experiment, they observed the students from the United States attended more to the mouth when watching videos in Catalan and Spanish -languages they were not skilled at- compared to the videos in English. The students from Barcelona (who did not know much of English) attended to the mouth when watching the video in English compared to the videos in Catalan and Spanish.
Once they could prove low-level speakers look at the speakerʼs mouth to understand it better, researchers explored the main objective of the study in a second experiment: to analyse whether language skills affect the eye and mouth attention of the speaker when receiving a language in L2. They chose students of the UB with different levels of English, ranging from low levels to high levels, and carried out the first experiment, by making comprehension questions after having seen the stories, to guarantee the level of participants corresponded to what they understood in the videos. “We thought that, the higher the level of the students, the better they would understand the videos and therefore, they would not need to look at the mouth of the speaker. However, all the students, regardless of their level, attended to the speakerʼs mouth half of the time, like in the first experiment. Therefore, we saw that even proficient L2 learners rely on the support of audiovisual elements (mouth) to understand the language”, notes Birulés.
The importance of audiovisual speech perception
Previous studied had showed that, whenever there is noise or a message is hard to understand, the face of the speaker helps us to understand the message. In these situations, we pay attention to the speaker and try to see the mouth in order to get audiovisual perceptions, such as articulatory movements that fit with the sounds temporally and acoustically.
The conclusions of the study suggest that we use audiovisual speech perception of the mouth to understand the message. “The obtained results did not only prove that the attention to the speakerʼs mouth increases whenever the voice process is difficult, but also highlighted that the same happens in the case of second language speakers, regardless of how skilled at the language they are”, concludes Birulés.