SAPIR - D3.1 - Common Schema for Feature Extraction

In this report we define a representation formalism for describing multimedia documents containing any combination of video, still images, music, speech, and text. A document description in this formalism includes metadata (author, title, etc.), as well as the results of automatic feature extraction for use in indexing, search, and browsing. By defining a single representation format that covers all media, we intend to support cross-media search; for example, an image similarity search might retrieve both videos and still images; and a keyword search on titles might receive documents of all media types. The representation is based on the MPEG-7 standard, with extensions to cover media, features, and metadata not covered by the standard. MPEG-7 provides a rich vocabulary for describing document structure and content, and its status as a standard means that SAPIR will be interoperable with other multimedia management systems. The SAPIR-specific extensions are defined in such a way as to preserve this interoperability. The report describes project activities undertaken as part of task T3.1