| |
To reach our objectives we will engage in a variety of collaborative research and prototyping activities.
An important part of our success will be the close collaboration among researchers within and across workpackages.
The Human-Computer Interfaces WP will benefit from work in other workpackages including the Single Modality Processing WP,
the Cross-Modal Integration WP, the Machine Learning for Multimedia Content WP and the Communication and Representation of Data,
Meta-data and Semantics WP. The research performed in this WP will contribute directly to the Integration WP.
The research work for this workpackage for the first 18 months consists of the following collaborative tasks:
|
|
- Task 1: Evaluation of the state-of-the-art in search of "low-hanging fruits:"
|
|
|
Partners will evaluate the current state-of-the-art in the area of human-computer interfaces for multimedia information retrieval.
The partners will then identify the most promising research directions and dynamically update their research agenda to reach the workpackage and NoE objectives.
|
|
- Task 2: Interactive Querying and Visualization of Query Results
|
|
|
Often results are thousands of hits per query. The query needs to be further refined by the user in an interactive way.
Our goal is to present these results in a structured way that takes into account the user's preference and past interaction with the system.
We will design and experiment with an interactive user interface that explores multimedia interfaces. |
|
- Task 3: Speech and Natural Language Interfaces for Q&A and the Web
|
|
We will focus our research work on:
(1) the integration of inputs from various sources (including different modalities, user profiles and past user history)
(2) the presentation of search results using multimedia output (text, audio, graphics)
(3) the interaction (dialogue) manager that combines the semantic information and decides what is the next step in interacting with the user.
|
|
- Task 4: Content Description and Adaptation Supporting Mobility
|
|
Specifically the work will focus on the following three research areas:
Content Description:
Data structures and ontologies for describing rich (multimedia) content in mobile environment will be created.
The creation of the models for adaptive contents is one of the key focuses.
Content Adaptation:
Tools for the adaptation of rich heterogeneous content to mobile terminals and for local semantic filtering and
adaptation will be created.
In addition to differences between various devices and networks, contextual information and preferences of users will be
taken into account in the adaptation.
This poses challenges also to content description including semantic information.
Content Creation and Annotation:
Various tools for producing multimedia content to/from mobile platforms are developed.
The focus is on intelligent methods for image and video processing, semantic filtering, adaptation and visualisation
in order to send/receive this content over low bandwidths.
|
|