CaTe: A Caption Template calculation program
In a joint project with the Australian Captioning Center, we
explore the use of MPEG video and audio analysis software to support
captioners (caption = subtitles for the deaf). To that end, we use
MPEG Maaate, the Mediaware
Systems video analysis library and some additional heuristic rules
to create a segmentation of the audio track that resembles closely to
the blocks that captioners produce for filling in the text. We thus
create blank caption files (EBU file format) which can be uploaded
into any existing captioning software to enter the caption text
itself.
Sports Highlights extraction program
CSIRO digital media information systems researchers have used MPEG
Maaate functionality in the development of a system
to extract sports highlights from sports footage. The system uses
both audio and video features of footage to determine segments
containing a highlight such as scoring a goal or an attack. It uses
crowd cheers detected via segmentation of noisy regions plus camera
operations such as pan and zoom to determine the highlights with every
sport having a different set of heuristic rules that determine a
highlight.
More application ideas:
- Automatic segmentation of large sound recordings e.g. story segmentation of news
- Sound search engine based on specific similarities to a given sample file
- Automatic classification of large sound file collections into specific classes e.g. different animals
- Browsing large sound file collections based on similarities or sound models
- Support for sound editing tools e.g. for filtering out crazy noises
Here's a radio interview of Silvia about the use of audio analysis technology for online music searching:
|