At SAIL LABS all key technologies are developed with a vision of continuous innovation at the leading edge of linguistic and IT research.

SAIL LABS currently participates in the following projects dealing with language and interactivity. The projects involve technology vendors, system integrators and users to develop new technologies and provide the technology vendors with immediate feedback from participating user groups regarding usability and applicability of the new technology.

Measurable Intelligent and Reliable Semantic Extraction and Retrieval of Multimedia Data
Multimedia data has a rich and complex structure in terms of inter- and intra-document references and can be an extremely valuable source of information. However, this potential is severely limited until and unless effective methods for semantic extraction and semantic-based cross-media exploration and retrieval can be devised.

MISTRAL will extract a large variety of semantically relevant metadata from one media type and integrate it closely with semantic concepts derived from other media types. Eventually, the results from this cross-media semantic integration will also be fed back to the semantic extraction processes of the different media types so as to enhance the quality of the results of these processes. MISTRAL will focus on most innovative, semantic-based cross-media exploration and retrieval techniques employing concepts at different semantic levels.

MISTRAL addresses the specifics of multimedia data in the global, networked context employing semantic web technologies. The MISTRAL results for semantic-based multimedia retrieval will contribute to a significant improvement of today’s human-computer interaction in multimedia retrieval and exploration applications. New types of functionalities include but are not limited to

• cross-media-based automatic detection of objects in multimedia data: For example, if a video contains an audio stream with barking together with a particular constellation of video features, the system can automatically consider the features in the video as an object “dog”.
  
  
• semantic-enriched cross-media queries: A sample query could be “find all videos with a barking dog in the background and playing children in the foreground”.
  
  
• cross-media synchronisation: The idea is to synchronize independent types of media according to the extracted semantic concepts. For example, if users see somebody walking in a video, they should also hear footfall from an audio.

SAIL LABS has been chosen as research partner in its area of expertise 'speech recognition'.

For more information please visit mistral-project.tugraz.at.

Funded by:

                       


                                

Retrieval of Video and Language for the home User in an Information Society

REVEAL THIS addresses a basic need underlying content organisation, filtering, consumption and enjoyment by developing content programming systems that will help European citizens keep up with the explosion of digital content scattered over different platforms (radio, TV, World Wide Web, etc), different media (speech, text, image, video) and different languages. People should be spending most of their leisure time enjoying the content, not searching for it.

REVEAL THIS aims at developing content programming technology able to capture, semantically index, categorise and cross-link multiplatform, multimedia and multilingual digital content, as well as provide the system user with semantic search, retrieval, summarisation and translation functionalities.

For more information please visit www.reveal-this.org.

Funded by:

This project aims to facilitate common procedures of archiving and retrieval of audio-visual material.The participating organisations in this project come from various countries like France, Belgium, Austria, Switzerland, and Greece.The objective of the project is to develop and integrate a robust unrestricted keyword spotting algorithm and an efficient image-spotting algorithm specially designed for digital audio-visual content, leading to the implementation and demonstration of a practical system for efficient retrieval in multimedia databases. Specifically, a system will be developed to automatically retrieve images, video, and speech frames from an audio-visual database based on keywords entered by the user through keyboard or speech.

Today, a vast amount of information is accumulated in the form of video, pictures, and audio, which does not lend itself to automated searching. To improve the usability of these invaluable resources, indexing techniques are required, which are currently very expensive and time-consuming tasks mainly carried out manually by experts. In view of the expansion of the digital television and of video-based communications and related applications the need for an editor-like tool that allows the user to see/hear, select/modify and search audio-visual databases becomes indispensable. Although some European projects are addressing the issue of automated indexing of audio-visual material based on subtitles and speech recognition, the problem of locating important video clips based on their image contents has not been addressed. CIMWOS will use a dual audio and visual approach to locate important clips within multimedia material employing state-of-the-art algorithms for both image and speech recognition.

The CIMWOS system will be a powerful tool in the hands of the world of media and television, video, news broadcasting, show business, advertisement, and any organisation that produces, markets and/or broadcasts video and audio programmes. It will facilitate common procedures of retrieving audio-visual material during a research, a production of a documentary, etc. Utilizing the vast amounts of information accumulated in audio and video, the CIMWOS system will become an invaluable assistant in promoting the re-use of existing resources and cutting down the budgets for new productions.

To find out more about the CIMWOS project go to www.xanthi.ilsp.gr/cimwos/.

Funded by:

V-Man: The Virtual Man  Project
SAIL LABS Technology is part of the ambitious V-Man consortium that aims to develop an intuitive system allowing non-computer specialists to create, animate, control and interact with a new generation of 3D virtual characters: the V-Men. These autonomous characters are intended for use in interactive media such as games and virtual reality as well as for special effects in film and television.

The project will bring together state-of-the-art video game, research and industrial 3D technologies allowing realistic simulation of body and clothes appearance, facial expressions, and real-time physics. Thus, the V-Man will be able to adapt its behavior to its environment, interact with its environment, and understand intuitive high-level user commands.

The V-Man product will be available as a stand-alone virtual reality application that exports animation in standard formats, a plug-in for the computer graphics applications, and a C++ toolkit allowing developers to populate their visual simulations or videogames with realistic characters. The V-Man system features realistic simulation of body and clothes appearance, facial expressions, and real-time physics. A V-Man is able to walk on any kind of terrain, to go upstairs, downstairs, to calculate paths in order to avoid obstacles, and to adapt his movements and actions to his environment. Transitions between movements are accomplished with innovative combination of motion blending algorithms, animation sampling methods and real-time physical simulation of the body. Physical character animation, where dynamics and animation is blended continuously allows V-Men synthesising motion at runtime depending on their environment, their task and their physical parameters.

The V-Man consortium is composed of six complementary organisations among the top-ranking ones in their respective fields of activity and based in five different European countries, thereby securing the dissemination and penetration of the project results across Europe.

End-users, coming from complementary fields of activity (simulation, television), will provide a perfect industrial setting for specifying and evaluating different V-Man versions on various real-world problems: each end-user partner will validate the software on a pilot application while providing valuable information for the elaboration of best practice and marketing materials.

With regards to the V-Man project, SAIL LABS Technology will provide components and solutions for the next generation language technology products with emphasis on natural language applications. This will involve next generation speech processing, multilingual document processing; multimedia content processing, multi- modal interfaces and dialogue systems.

Funded by: