WP 10: Human-Computer Interfaces for Multimedia Retrieval  
WorkPackage Leader:  
Professor Alexandros Potamianos, Technical University of Crete, Home Page  
Objectives  

Multimedia information retrieval via an interactive human-computer interface is a complex task that requires feedback from the user and a complex negotiation between the user and the machine. We propose to research, design and build natural and efficient human-computer interfaces for performing multimedia information retrieval tasks that allow for negotiation (dialogue) between the user and the system. Objectives of this work include:
  1. Create semantic representations (ontologies) and semantic structures that are generalizable across languages, media and modalities.
  2. Investigate various input modalities and information sources (speech, natural language text, query text, eye-tracking, user preferences, past user behavior) and their synergies in multimedia information retrieval tasks.
  3. Investigate various output modalities (text, graphics, video, audio) to create a visualization of multimedia query results and help the user navigate a multimedia database.
  4. Build natural and efficient multimodal multi-turn human-computer interfaces (combining 1-3) that advance the state-of-the-art and demonstrate the main concepts of our research.