Mutual influences between epistemic intonation and co-speech gesture in online language comprehension

Mutual influences between epistemic intonation and co-speech gesture in online language comprehension

Evangelia Kiagia1, Joan Borrás-Comes1,2  and Pilar Prieto1,3

1. Universitat Pompeu Fabra

2. Universitat Autònoma de Barcelona

3. ICREA

While a number of previous studies have proposed that (iconic) gestures and speech interact mutually and obligatorily during online processing (e.g., Kelly et al. 2003; 2010) little is known about the mutual and bidirectional influences between pragmatic prosody and gestures. Previous studies have shown how certain intonation patterns and gestures encode the speaker's commitment to the proposition and the speaker's agreement with its interlocutor (Borrás-Comes & Prieto 2011).

Experiment 1 presented participants with a set of gesture primes expressing levels of speaker agreement and commitment and asked them to produce a target phrase. Pilot results with 10 participants show that the set of gesture primes elicited different levels of speaker agreement and commitment intonation patterns, confirming a direct influence of gestures on the corresponding intonation patterns. Experiment 2 consisted of an eye tracking visual search experiment in which participants saw images with epistemic gestures while listening to neutral sentences produced with a set of intonation patterns carrying different levels of speakers agreement and commitment. Pilot results from the two experiments demonstrate the bidirectional and obligatory influences between intonation and gesture, and more specifically, that epistemic intonation and gestures form a semantically integrated system in online language comprehension.