|
|
||||||
|
|
||||||
|
|
|
|
||||
|
|
|
|
Salvador Soto Faraco ICREA Research Professor
Parc Cientific
de Barcelona and Dept.de Psicologia Bàsica Universitat
de Barcelona Hospital
Sant Joan de Déu Edifici
Docent C/Santa Rosa,3957, planta 4ª 08950 Esplugues
de Llobregat - Barcelona Spain E-Mail: salvador.soto@icrea.es Tef. +34 93 600 97 69 Fax.+34 93 600 97 68 |
Interests Publications Research lines |
||
|
|
||||||
|
|
|
|||||
|
|
|
I
am interested in the neural and cognitive basis of human perception and
attention processes in multisensory environments.
That is, the question of how our brain organizes, represents, and selects the
information that arrives from the different sensory modalities (audition, somatosensation, vision, olfaction, etc). Almost
every aspect of our everyday lives involves combining information from more
than one sense, like when engaging in a face-to-face conversation (as we
attend the sounds and the lip movements of the speaker), adopting new
postures when we move about, or quickly locating the origin of an unexpected
event. Several of my research projects address basic questions about the
mechanisms responsible of binding sensory information during speech
perception, stimulus localization, motion perception, or representation of
body schema. I use a range of different methods, including behavioural
experimentation / psychophysics, neuropsychological approach with
brain-damaged patients, electrophysiological methods (ERPs)
and, in collaboration with other centres, functional Magnetic Resonance
Imaging (fMRI). My laboratory also addresses
applied research targeted at specific real-world questions such as how to
improve consumer experience during the use of certain products, or to find
out how to enhance the perceived naturalness of materials. These
research projects receive funding from public and private sources, including.
Ministerio de
Educación y Ciencia (MEC), Departament d' Universitats, Recerca i Societat
de |
||||
|
|
|
|||||
|
|
|
Articles
in refereed academic journals Deco, G., Scarano, L. & Soto-Faraco,
S. ( in press, Aug 2007). Weber's Law in
Decision-Making: Integrating Behavioral Data in
Humans with a Neurophysiological Model. Journal of Neuroscience Alsius,
A., Navarra, J. & Soto-Faraco, S. (in press, Aug
2007). Attention to touch weakens audiovisual speech integration. Exp. Brain Res. Lopez-Moliner,
J. & Soto-Faraco, S. (In press, Jun 2007).
Vision affects how fast we hear sounds move. Journal of Vision Azañón,
E. & Soto-Faraco, S. (In Press, Jun 2007).
Alleviating the ‘crossed-hands' deficit by seeing uncrossed rubber hands. Exp. Brain Res. Weikum
W.M., Vouloumanos, A., Navarra, J. Soto-Faraco, S., Sebastián-Gallés,
N. & Werker, J. F. (2007). Visual language
discrimination in infancy. Science ,
316 (5828): 1159 Vatakis,
A., Navarra, J., Soto-Faraco, S., & Spence, C.
(In Press, 2007). Temporal recalibration during asynchronous audiovisual
speech perception. Exp. Brain Res .
[DOI: 10.1007/s00221-007-0918-z] Soto-Faraco,
S. & Alsius, A. (2007). Conscious access to the
uni-sensory components in a cross-modal illusion. Neuroreport, Mar 5; 18 (4): 347-50
Sinnett,
C., Spence, C., & Soto-Faraco, S. (in press,
July, 2006). Visual dominance and attention: The Colavita
effect revisited. Perception & Psychophysics
Navarra, J., Soto-Faraco, S.
& Spence, C. (2007). Adaptation
to audiotactile asynchrony. Neuroscience Letters, Feb 8; 413(1) :72-6. Sanabria,
D., Soto-Faraco, S. & Spence, C. (In Press,
2006). Spatial attention modulates audiovisual interactions: The case of
apparent motion. Journal of Experimental
Psychology; Human Perception and Performance Soto-Faraco, S., Navarra, J., Voloumanos, A., Sebastián-Gallés,
N., Weikum, & Werker,
J. F. (2007). Discriminating languages by speechreading. Perception
and Psychophysic , 69 (2), 218-231. Sinnett,
C., Juncadella, M. , Rafal, R., Azañón, E. &
Soto-Faraco, S. (2007). A
dissociation between visual and auditory hemineglect:
Evidence from temporal order judgements. Neuropsychologia , 45(3) , 552-560 Sanabria,
D., Spence, C., & Soto-Faraco, S. (2007).
Perceptual and decisional contributions to audiovisual interactions in the
perception of apparent motion: A signal detection study, Cognition, 102 (2):299-310. Soto-Faraco,
S., Foxe, J. J. & Wallace, M. T. (2005). Multisensory
processes. Experimental Brain Research ,
166 (3-4), 287-288 Navarra, J., Zampini, M., Soto-Faraco, S., Humphreys, W. &
Spence, C. (2005). Exposure
to audiovisual asynchrony modulates the temporal window for audiovisual
integration. Cognitive Brain Research ,
25 , 499-507 Sinnett,
S., Costa, A. & Soto-Faraco, S. (2006).
Manipulating Inattentional Blindness across sensory
modalities. Quarterly Journal of
Experimental Psychology – Section A , 59 (8), 1425–1442 Navarra, J. & Soto-Faraco, S. (2007). Hearing lips in a second language:
Visual articulatory information enables the
perception of L2 sounds. Psychological
Research , 71 (1),
4-12 Alsius,
A., Navarra, J., Campbell, R. & Soto-Faraco, S.
(2005). Audiovisual speech integration falters under high attention demands. Current Biology , 15 (9), 839-843 Soto-Faraco,
S., Sinnett, C., Alsius,
A. & Kingstone, A. (2005). Eyegaze
orienting to tactile targets. Psychonomic
Bulletin & Review , 12
(6):1024-31 Navarra, J., Sebastián-Gallés, N. & Soto-Faraco,
S. (2005). The perception of second language sounds in early bilinguals: new
evidence from an implicit measure. Journal
of Experimental Psychology: Human Perception and Performance , 31 (5), 912-918 Toro, J. M., Sinnett, S. & Soto-Faraco,
S. (2005). The consequences of diverting attention within and across sensory
modalities on statistical learning. Cognition,
97 (2), B25-B34 Soto-Faraco,
S. (2005). Book Review: ‘Crossmodal space and crossmodal attention'. European
Journal of Cognitive Psychology, 17 (6) , 882-885
Sanabria,
D., Soto-Faraco, S., & Spence, C. (2004).
Exploring the role of visual perceptual grouping on the audiovisual
integration of motion. Neuroreport , 15 (18), 2745-2749 Sanabria,
D., Soto-Faraco, S., Chan, J., & Spence, C.
(2005). Intramodal perceptual grouping modulates multisensory integration: Evidence from the crossmodal dynamic capture task. Neuroscience Letters, 377 , 59-64 Sanabria,
D., Soto-Faraco, S., & Spence, C. (2005).
Assessing the influence of visual and tactile distractors
on the perception of auditory apparent motion. Experimental Brain
Research, 166
(3-4),
548-558 Soto-Faraco, S., Navarra, J., Sinnett, S., & Alsius, A.
(2004). La re-unificación de los sentidos. Reseña del libro: The Handbook of Multisensory Proceses. Cognitiva, 16 , 257-259. Sanabria, D., Soto-Faraco, S.,
& Spence, C. (2005). Spatiotemporal
interactions between audition and touch depend on hand posture. Experimental Brain Research, 165 (4), 505-514 Soto-Faraco,
S., Spence, C. & Kingstone, A. (2005).
Automatic capture of auditory apparent motion. Acta Psychologica,
118 , 71-92 Soto-Faraco, S. (2004). Crítica
de libros: The Handbook of Multisensory Processes. Revista de
Neurología, 39 (7),
700. Soto-Faraco, S., Ronald, A., & Spence, C.
(2004). Tactile Selective Attention and Body Posture:
Assessing the Multisensory Contributions of Vision
and Proprioception. Perception & Psychophysics , 66 (7), 1077-1094 Soto-Faraco,
S., Morein-Zamir, S., & Kingstone,
A. (2005). On audiovisual spatial synergy: The fragility of the phenomenon. Perception & Psychophysics, 67 (3):444-57 Soto-Faraco,
S., Spence, C. & Kingstone, A. (2004).
Congruency Effects between Auditory and Tactile Motion: Extending the
Phenomenon of Crossmodal Dynamic Capture Cognitive, Affective, and Behavioral
Neuroscience, 4 ,
208-217 Sanabria, D., Soto-Faraco, S., Chan, J. S., & Spence, C.
(2004). When does visual perceptual grouping affect multisensory integration? Cognitive, Affective, and Behavioral Neuroscience,
4 , 218-229 Austen, E., Soto-Faraco, S., Enns, J. T., & Kingstone, A. (2004). Mislocalizations
of touch to a fake hand Cognitive,
Affective, and Behavioral Neuroscience, 4,
170-181 Soto-Faraco, S., Navarra, J.,
& Alsius, A. (2004). Assessing Automaticity in Audiovisual Speech Integration: Evidence
from the Speeded Classification Task. Cognition,
92 ,
B13-B23 Soto-Faraco,
S., Spence, C., Lloyd, D., & Kingstone, A.
(2004). Moving multisensory research along: Motion
perception across sensory modalities. Current
Directions in Psychological Science, 13 (1), 29-32 Soto-Faraco,
S., Spence, C. & Kingstone, A. (2004). Crossmodal dynamic capture: Congruency effects in the
perception of motion across sensory modalities. Journal of Experimental Psychology: Human Perception and Performance,
30 (2):330-45 Soto-Faraco,
S., Chan, J., Kingstone, A. & Spence, C.
(2003). Multisensory contributions to motion perception. Neuropsychologia , 41 (13), 1847-1862 Costa, A. & Soto-Faraco, S.
(2004). La vieja nueva
ciencia y la tesis de modularidad. Comentario sobre
Escera. Cognitiva
, 16 (2),
177-185 Morein-zamir,
S., Soto-Faraco, S., & Kingstone,
A. (2002). The temporal capture of vision by audition. Cognitive Brain Research, 17 , 154-163 Soto-Faraco,
S., Spence, C., Fairbank, C., Kingstone,
A., Hillstrom, A., & Shapiro, K. (2002). A crossmodal Attentional Blink
between vision and touch. Psychonomic
Bulletin & Review ,
9, 731-738. Kingstone,
A., Dazinger, S., Langton, S. & Soto-Faraco, S. (2002). Attentional capture:
Biological relevance, multisensory stimulation, and
consciousness. A commentary on Ruz & Lupiañez. Psicológica, 23, 343-346. Caclin,
A., Soto-Faraco, S., Kingstone,
A. & Spence, C. (2002). Tactile ‘capture' of audition. Perception & Psychophysics , 64 , 616-630. Soto-Faraco,
S. & Sebastián-Gallés, N. (2002). Identity and
similarity in repetition deafness. Psicológica, 23 , 189-208. Soto-Faraco, S., Lyons, J., Gazzaniga, M. S., Spence, C. & Kingstone, A.
(2002). The ventriloquist in motion: Illusory capture of
dynamic information across sensory modalities. Cognitive Brain Research , 14 , 139-146 Soto-Faraco,
S. and Spence, C . (2002). Modality specific
auditory and visual temporal processing deficits. Quarterly Journal of Experimental Psychology, 55A , 23-40 Soto-Faraco,
S. and Spence, C. (2001). Spatial modulation of repetition deafness and
repetition blindness. Quarterly Journal
of Experimental Psychology, 54A , 1181 - 1202 Spence, C., Shore, Soto-Faraco,
S., Sebastián-Gallés, N. and Cutler, A. (2001).
Segmental and supra-segmental mismatch in lexical access. Journal of Memory and Language , 45 , 412-432 Soto-Faraco,
S. and Sebastián-Gallés, N. (2001). The effect of
acoustical mismatch and selective listening on auditory repetition deafness. Journal of Experimental Psychology: Human
Perception & Performance, 27 (2), 356-369 Soto-Faraco,
S. (2000). An auditory repetition deficit under low memory load. Journal of Experimental Psychology: Human
Perception & Performance ,
26 (1), 264-278. Sebastián-Gallés,
N. and Soto-Faraco, S. (1999). On-line processing
of native and non-native phonemic contrasts in early bilinguals. Cognition, 72 , 111-123.
Book
chapters Soto-Faraco,
S., Kingstone, A., & Spence, C. (2006). Integrating
motion information across sensory modalities: The role of top-down factors.
In S. Martínez-Conde et al. (Eds).
Visual Perception, series Progress in
Brain Research (vol 155, pp.273-286). Spence, C., Sanabria, D., & Soto-Faraco,
S. (2007). Intersensory Gestalten and Crossmodal Scene Perception. In K. Noguchi (Ed.), The psychology of beauty and Kansei:
New horizons of Gestalt perception Soto-Faraco, S., Navarra, J., Alsius, A., & Campbell, R.
(2005). Efectos de la atención en la integración audiovisual del habla. (pp.
75-85). Atencion y Procesamiento . València:
Universitat de València y
Fundación Universidad-Empresa. Soto-Faraco, S. & Sanabria,
D. (2005). Multisensory
interactions in motion perception. In Diane T. Rosen (Ed.). (pp. 163-185). Trends in Experimental Psychology Research .
Nova Science Publishers Inc.: NY Soto-Faraco,
S. & Kingstone, A. (2004). Multisensory
integration of dynamic information. In The Handbook of Multisensory
Processes. G. Calvert, C. Spence & B. Stein (Eds.). (pp. 49-68). The MIT
Press. Sebastian-Galles, N. & Soto-Faraco, S. (2000). L'atenció.
In Processos Psicològics Bàsics . H. Boada
& A. Caparrós (Eds.).
Pulicacions Univesitat Oberta de Catalunya. |
||||
|
|
|
|||||
|
|
|
Audiovisual
integration during speech perception. Verbal
communication is a multisensory phenomenon because
the consequences of the speech act are available to the eye (lip-movements) as
well as to the ear (acoustic stream). The integration of audiovisual
information in speech has often been cited as the prototypical example of the
automatic and mandatory (i.e., “attention-free”) nature of multisensory binding. These multisensory
mechanisms for communication seem to work as well in animals (such as
monkeys) as well as in very young infants. I am interested in audiovisual
speech as a model for multisensory integration
processes in general, and in the role of audiovidual
integration in multilingual environments in particular. The outcome of this
research project can prove valuable to understand how audiovisual integration
can help hearing-impaired people, aid in the design of more effective
audio-visual communication technology (i.e., video-conferencing), and devise
improved strategies for learning second languages. Multisensory
integration processes in the domain of time Sensory
experiences such as the sight, sound and touch when examining an object
appear to us as simultaneous, but despite physical simultaneity (the moment
at which information is generated), information arrives at different times to
our different sensory areas in the brain. This is a natural consequence of
differences in physical and neural transmission times of distinct types of
energy. I am interested in the
mechanisms whereby the brain couples sensory sensations in time. For example,
sounds can “attract” the moment at which a visual event is perceived to occur
in time by a small but significant amount, or exposure to slightly
desynchronised audiovisual events (such as when we watch TV programs
broadcasted life from a far away location) can induce the perceptive system
to re-calibrate sensory modalities in time. Multisensory
contributions to motion perception Given that
the majority of our experience with the environment involves dynamic events
(i.e., most of what we perceive is moving around), an important question
concerns the extent to which the principles derived from the study of multisensory integration with stationary stimuli also
apply to moving stimuli. A key finding to emerge from this line of research
is that our perception of stimulus movement in one modality is frequently,
and unavoidably, modulated by the concurrent movement of stimuli in other
sensory modalities. We have found that visual motion has a particularly
strong influence on the perception of auditory and tactile motion. These
behavioural results are now being complemented by the results of neuroimaging studies that highlight the existence of both
modality-specific motion processing areas as well as areas involved in the
processing of motion in more than one sense. The
representation of tactile space The way in which our
brains keep track of where our body parts are (i.e., depending on what postures
we adopt) is a particularly interesting example of the importance that multisensory integration processes have in perception.
The primary somatosensory area of the human brain
(S1), a strip of cortex located in the anterior part of the parietal area, is
neatly arranged to follow the topography of the body surface (the classic
homunculus first described by Penfield & Rasmussen, 1950). Activity in S1
signals the location of tactile stimulation in terms of an anatomically-based
map (i.e., a somatotopic map). However, in order to
localize and act upon sensory inputs in the environment, most of our somatosensory experience must be referred to spatial
locations defined according to an external frame of reference. For example,
as the body can adopt different postures, a tactile sensation at the hand
could arise from anywhere within reachable space. Being able to rapidly
localize the source of tactile stimulation could become critical if we were
to react to the origin of tactile information, and to do so the brain must
compute the location of tactile events in terms of an external spatial
reference frame. This more elaborate representation of ‘tactile’ space can
only be achieved through the combination of somatosensory
inputs with information about body posture through proprioception
and/or vision. We study the nature of the mechanisms that help re-map tactile
information as a function of body posture. Attention
in multisensory environments
Probably everyone has experienced the situation of walking by a friend in the
street without even noticing (with the consequent embarrassment later on)
because we were concentrated on something else. The consequences of
inattention can be sometimes dramatic as, for instance, many road accidents
are blamed on not “seeing” something because of lack of attention. We are
interested in investigating the consequences of varying the focus of
attention (or lack thereof) in multisensory
situations applying several converging research strategies. In some of the
studies we used dual task situations within- and across-modalities during
perceptual processes such as word recognition, speech segmentation, or
audiovisual integration, we have also addressed the
temporal course of attention, and finally, the spatial distribution of
attention within and across sensory modalities. In our research we test both
healthy observers as well as neurological patients with severe attention
deficits such as spatial hemineglect. Overall, the
results of this line of research have led us to qualify the traditional claim
that different sensory modalities are served by separate and independent
attention systems. Instead, our findings have helped highlight potential
links in attention across sensory modalities in several situations, as well
as to delineate the boundary conditions where these interactions are less
likely to occur. The outcome of this research line is relevant to help design
more efficient (i.e., less attention-demanding) human-machine interfaces to
allow, for example, safer car driving or working environments. The
perception of naturalness We instinctively know whether a
material (cloth, wood, steel, …) is natural, or a
synthetic mimic. But the key factors responsible for this perception have yet
to be identified: this is the primary objective of this project. The
perception of naturalness is dominated, in most cases, by visual appearance
and touch. Basic visual sensory information, such as colour and gloss, and
other visual triggers, such as shape and size is often sufficient to
differentiate between natural and synthetic materials. Touching the material
serves to reinforce the initial visual perception: here tactile information
from cutaneous pressure sensitive and thermal
sensory transducers, as well as kinesthetic data,
provides the requisite sensory input. This project aims to understand how
these sensory data streams are processed by the relevant neural networks and
how they contribute to the cognitive processes associated with the perception
of naturalness. This understanding will take account of the effect of
contextual information on this perception (i.e. the interplay between the
individual senses and the relationship between the material and its
environment). Establishing the chain of perception for naturalness based on
these sensory inputs will enable relationships between the physical
attributes of the material and the neural and cognitive processes to be
identified; information that will be used in the project to develop
mathematical models to predict the perception of naturalness for a range of
materials. The project involves a multidisciplinary team, with strong
expertise in the areas of physical measurement, instrumentation, cognitive
neuroscience, psychology and mathematical modelling. |
||||
|
Links |
|
|
||||
|
|
|
International Multisensory
Research Forum (IMRF) |
||||
|
|
|
|
||||
|
|
||||||
|
|
|
|
||||
|
|
||||||
|
p |
||||||
|
|
||||||
