Search In Audio Visual Content Using Peer-to-peer IR
Period:January 1, 2007 - July 1, 2011
SAPIR extends the power of web searches beyond centralized text and metadata searches to include distributed audio-visual content.
Today, Web searches are dominated by search giants such as Google, Yahoo, or MSN that deploy a centralized approach to indexing and utilize text-only indexes enriched by page rank algorithms. Consequently, while it is possible to search for audio-visual content, the search is limited to associated text and metadata annotations. Supporting real content-based, audio-visual search requires media-specific understanding and extremely high CPU utilization, which would not scale in today’s centralized solutions.
SAPIR aims at breaking this technological barrier by developing a large-scale, distributed P2P architecture that will make it possible to search audio-visual content using the query-by-example paradigm. Considering that a picture is worth a thousand words, an image taken by a cell phone can be used to find information. For example, search for a monument using its photo or search for a full song using its melody. Combining these search hints with optional metadata annotations and user and social networking contexts will provide the next level of search capabilities with precise retrieved results.
Our vision is to conduct innovative research that will lead to a technology where end-users are peers that can produce audio-visual content from their mobile devices. This content will be indexed by super-peers across a scalable P2P network to enable content searches in real-time, while respecting IPR and protecting against spam. To this end, SAPIR brings experts in audio-visual content understanding in the areas of text, audio, image, video, and music analysis.
A common framework for feature extraction from all media contents will be developed for similarity search and ranking along all supported media. To address scalability issues, we will develop a P2P architecture where features can be extracted in one peer and pushed to an indexing peer. The P2P architecture will provide a scalable indexing structure that can be used for multi-feature search. Caching techniques will be developed to increase system performance. To further improve audio-visual retrieval and navigation, SAPIR’s consortium combines experts in mobile device technology, along with with experts in social networking and IPR to enable a secure and trusted environment.
This technology can provide a significant advantage to the European community over existing, centralized, text-only search engines and can be applied to various fields such as tourism, government services, healthcare, and more.