MPEG-7
Multimedia Content Description Interface

by
Jozsef Vass
2/2/98

Table of Contents

1) Overview	2) Introduction	3) History	4) MPEG-7 Documents
5) Schedule	6) MPEG-7 Objectives	7) Scope of MPEG-7	8) MPEG-7 Requirements
9) MPEG-7 Applications	10) Research Directions	11) Case Study - QBIC	12) Case Study - Finding Images/Video Archives

Not a compression standard
MPEG-7 will be associated with the indexing of the available audio-visual (AV) information
Before usage, information needs to be located -> quickly and efficiently search for various types of multimedia information
Search for textual information exists (WWW search engines) but locating multimedia material is very difficult
Information about content -> metadata
Why MPEG-7? -> MPEG-5 would continue the series and MPEG-8 would give a binary pattern

Call for Proposals	November 1998
Working Draft	July 1999
Committee Draft	March 2000
Draft International Standard	July 2000
International Standard	November 2000

Allow fast and efficient searching for multimedia material of user's interest
Specify a standard set of description schemes and descriptors to describe various types of multimedia information
Specify encoding of description schemes and descriptors
Descriptors are associated with the content itself
Descriptors must be meaningful in the context of application
MPEG-7 data can be located anywhere
Descriptors are independent of the encoding of the content
Any type of AV material can be retrieved by any type of query material

Requirements are divided into 1) common audio and visual requirements, 2) visual requirements, and 3) audio requirements

Query Classes: MPEG-7 shall support audio and visual descriptors that allows different types of queries:
-- Textual Description Classes: keywords, title, author, structure, etc.
-- Visual Description Classes: color, texture, shape, volume, etc.
-- Audio Description Classes: frequency contour, timbre, harmony, textual content, etc.
Content-Based Retrieval: Support effective and efficient retrieval of multimedia based on their content
Types of Features: Various types of features ranging from low level to high level features
Feature Hierarchy and Scalability: Coarse-to-fine refinement of features
Distributed Databases: Support of simultaneous and transparent retrieval
Robustness to Information Errors and Loss
Ability to export existing databases into MPEG-7 format
Copyright Information

Application domains:

Possible applications:

Intelligent Multimedia Presentation: By using knowledge of context, application, location of user the system can adapt to the user (similar to Web personalizing)
Storage and Retrieval of Video Databases: Enormous interest to search end retrieve from existing huge radio and TV archives
Teleshopping: Humans remember better to visual information than textual. Internet catalogs could be searched much easier by using visual query.
Entertainment: For example Karaoke user interface can be made more convenient by query by example (sing part of song)
Medical: Physicians can search huge medical archives for similar symptoms by query by example

Combination of low level information (extracted automatically) with high level description (mostly obtained manually)
Data structures
How MPEG-4's AV objects can be extended to support the requirements of MPEG-7?
Language independence

Image-Based Visual Query Project
WWW-based
Retrieval is based on features (motion, spatial, visual, syntactic or semantic characteristics of objects)
Operating in compressed domain (MPEG, not progressive) \item More information: http://www.ctr.columbia.edu/~sfchang