Multimodal Interaction

By Dr. Ing. Sebastian Möller, Dr. Benjamin Weiss, Dr. Rahul Swaminathan and Dr. Ing. Jens Ahrens

Technische Universität Berlin

 

Week 1

• 
  Introduction
  
• 
  Medium vs. Modality
  
• 
  Multimedia and Multimodality
  
• 
  Modality Relations
  
• 
  Characteristics of Multimodal Systems
  
• 
  Introduction.
  
• 
  Speech Production
  
• 
  Hearing – Ear
  
• 
  Hearing – Perception
  
• 
  Introduction..
  
• 
  The Human Eye
  
• 
  Gestlat Perception
  
• 
  Resolution and Sensitivity
  
• 
  Depth Perception
  
• 
  Reading
  

Week 2

• 
  Introduction
  
• 
  Haptics
  
• 
  Smell
  
• 
  Taste
  
• 
  Memory
  
• 
  Motorsystem
  
• 
  Introduction
  
• 
  Processing Multiple Signals
  
• 
  Multimodal Dual-Tasks
  
• 
  Effects of Discongurent Signals
  
• 
  Relevance
  
• 
  Introduction 1
  
• 
  Introduction 2
  
• 
  Gesture to Space
  
• 
  Turn Taking
  
• 
  Conclusion
  

Week 3

• 
  Introduction
  
• 
  Overview
  
• 
  Automatic Speech Recognition
  
• 
  Emotion Recognition
  
• 
  Text Recognition
  
• 
  Introduction1
  
• 
  Icons
  
• 
  Text Generation
  
• 
  Text to Speech
  
• 
  Speech Generation
  

Week 4

• 
  Introduction
  
• 
  Multimodal Interactive Systems Development
  
• 
  Introduction . .
  
• 
  Virtual Reality
  
• 
  Introduction to Audio for Virtual Reality
  
• 
  Spatial Hearing
  
• 
  Dummy Heads
  
• 
  Individuality of HRTFs
  
• 
  Sterophony
  
• 
  Crosstalk Cancelation
  
• 
  Ambisonics
  
• 
  Sound Field Synthesis
  
• 
  Challenges with Projection-based Systems
  
• 
  Capturing of Sound Scenes
  
• 
  Closing Remarks