Video data management and information retrieval

409 Pages·2005·9.517 MB·English

by  Sagarmay Deb

(cid:1)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6)(cid:7)(cid:8)(cid:9)(cid:8)
(cid:10)(cid:8)(cid:11)(cid:8)(cid:12)(cid:4)(cid:13)(cid:4)(cid:11)(cid:9)
(cid:8)(cid:11)(cid:3)
(cid:14)(cid:11)(cid:15)(cid:5)(cid:16)(cid:13)(cid:8)(cid:9)(cid:2)(cid:5)(cid:11)
(cid:17)(cid:4)(cid:9)(cid:16)(cid:2)(cid:4)(cid:18)(cid:8)(cid:19)
Sagarmay Deb
University of Southern Queensland, Australia
(cid:14)(cid:17)(cid:10)(cid:6)(cid:20)(cid:16)(cid:4)(cid:21)(cid:21)
(cid:20)(cid:22)(cid:23)(cid:19)(cid:2)(cid:21)(cid:24)(cid:4)(cid:16)(cid:6)(cid:5)(cid:15)(cid:6)(cid:2)(cid:11)(cid:11)(cid:5)(cid:18)(cid:8)(cid:9)(cid:2)(cid:18)(cid:4)(cid:6)(cid:21)(cid:25)(cid:24)(cid:5)(cid:19)(cid:8)(cid:16)(cid:19)(cid:26)(cid:6)(cid:8)(cid:11)(cid:3)(cid:6)(cid:27)(cid:16)(cid:5)(cid:15)(cid:4)(cid:21)(cid:21)(cid:2)(cid:5)(cid:11)(cid:8)(cid:19)
(cid:2)(cid:11)(cid:15)(cid:5)(cid:16)(cid:13)(cid:8)(cid:9)(cid:2)(cid:5)(cid:11)(cid:6)(cid:9)(cid:4)(cid:25)(cid:24)(cid:11)(cid:5)(cid:19)(cid:5)(cid:12)(cid:26)(cid:6)(cid:9)(cid:2)(cid:9)(cid:19)(cid:4)(cid:21)(cid:6)(cid:2)(cid:11)(cid:6)(cid:9)(cid:24)(cid:4)(cid:6)(cid:25)(cid:26)(cid:23)(cid:4)(cid:16)(cid:8)(cid:12)(cid:4)
Hershey • London (cid:127) Melbourne (cid:127) Singapore
Acquisitions Editor: Mehdi Khosrow-Pour
Senior Managing Editor: Jan Travers
Managing Editor: Amanda Appicello
Development Editor: Michele Rossi
Copy Editor: Jane Conley
Typesetter: Jennifer Wetzel
Cover Design: Lisa Tosheff
Printed at: Yurchak Printing Inc.
Published in the United States of America by
IRM Press (an imprint of Idea Group Inc.)
701 E. Chocolate Avenue, Suite 200
Hershey PA 17033-1240
Tel: [phone]
Fax: [phone]
E-mail: [email protected]
Web site: http://www.irm-press.com
and in the United Kingdom by
IRM Press (an imprint of Idea Group Inc.)
3 Henrietta Street
Covent Garden
London WC2E 8LU
Tel: [phone]
Fax: [phone]
Web site: http://www.eurospan.co.uk
Copyright © 2005 by IRM Press. All rights reserved. No part of this book may be reproduced in
any form or by any means, electronic or mechanical, including photocopying, without written
permission from the publisher.
Library of Congress Cataloging-in-Publication Data
Video data management and information retrieval / Sagarmay Deb, editor.
p. cm.
Includes bibliographical references and index.
ISBN 1-59140-571-8 (h/c) -- ISBN 1-59140-546-7 (s/c) -- ISBN 1-59140-547-5 (ebook)
1. Digital video. 2. Database management. 3. Information storage and retrieval
systems. I. Deb, Sagarmay, 1953-
TK6680.5.V555 2004
006--dc22
2004022152
British Cataloguing in Publication Data
A Cataloguing in Publication record for this book is available from the British Library.
All work contributed to this book is new, previously-unpublished material. The views expressed in
this book are those of the authors, but not necessarily of the publisher.
(cid:1)(cid:2)(cid:3)(cid:4)(cid:5)(cid:6)(cid:7)(cid:8)(cid:9)(cid:8)
(cid:10)(cid:8)(cid:11)(cid:8)(cid:12)(cid:4)(cid:13)(cid:4)(cid:11)(cid:9)(cid:6) (cid:8)(cid:11)(cid:3)
(cid:14)(cid:11)(cid:15)(cid:5)(cid:16)(cid:13)(cid:8)(cid:9)(cid:2)(cid:5)(cid:11)
(cid:17)(cid:4)(cid:9)(cid:16)(cid:2)(cid:4)(cid:18)(cid:8)(cid:19)
(cid:28)(cid:8)(cid:23)(cid:19)(cid:4)(cid:6)(cid:5)(cid:15)(cid:6)(cid:29)(cid:5)(cid:11)(cid:9)(cid:4)(cid:11)(cid:9)(cid:21)
Preface ...........................................................................................................................vi
Sagarmay Deb, University of Southern Queensland, Australia
SECTION I:
AN INTRODUCTION TO VIDEO DATA MANAGEMENT AND INFORMATION RETRIEVAL
Chapter I
Video Data Management and Information Retrieval......................................................1
Sagarmay Deb, University of Southern Queensland, Australia
SECTION II:
VIDEO DATA STORAGE TECHNIQUES AND NETWORKING
Chapter II
HYDRA: High-performance Data Recording Architecture for Streaming Media .......9
Roger Zimmermann, University of Southern California, USA
Kun Fu, University of Southern California, USA
Dwipal A. Desai, University of Southern California, USA
Chapter III
Wearable and Ubiquitous Video Data Management for Computational Augmentation
of Human Memory..........................................................................................................33
Tatsuyuki Kawamura, Nara Institute of Science and Technology, Japan
Takahiro Ueoka, Nara Institute of Science and Technology, Japan
Yasuyuki Kono, Nara Institute of Science and Technology, Japan
Masatsugu Kidode, Nara Institute of Science and Technology, Japan
Chapter IV
Adaptive Summarization of Digital Video Data .............................................................77
Waleed E. Farag, Zagazig University, Egypt
Hussein Abdel-Wahab, Old Dominion University, USA
Chapter V
Very Low Bit-rate Video Coding....................................................................................100
Manoranjan Paul, Monash University, Australia
Manzur Murshed, Monash University, Australia
Laurence S. Dooley, Monash University, Australia
SECTION III:
VIDEO DATA SECURITY AND VIDEO DATA SYNCHRONIZATION AND TIMELINESS
Chapter VI
Video Biometrics............................................................................................................149
Mayank Vatsa, Indian Institute of Technology, Kanpur, India
Richa Singh, Indian Institute of Technology, Kanpur, India
P. Gupta, Indian Institute of Technology, Kanpur, India
Chapter VII
Video Presentation Model..............................................................................................177
Hun-Hui Hsu, Tamkang University, Taiwan, ROC
Yi-Chun Liao, Tamkang University, Taiwan, ROC
Yi-Jen Liu, Tamkang University, Taiwan, ROC
Timothy K. Shih, Tamkang University, Taiwan, ROC
SECTION IV:
VIDEO SHOT BOUNDARY DETECTION
Chapter VIII
Video Shot Boundary Detection.....................................................................................193
Waleed E. Farag, Zagazig University, Egypt
Hussein Abdel-Wahab, Old Dominion University, USA
Chapter IX
Innovative Shot Boundary Detection for Video Indexing ...............................................217
Shu-Ching Chen, Florida International University, USA
Mei-Ling Shyu, University of Miami, USA
Chengcui Zhang, University of Alabama at Birmingham, USA
Chapter X
A Soft-Decision Histogram from the HSV Color Space for Video Shot Detection.......237
Shamik Sural, Indian Institute of Technology, Kharagpur, India
M. Mohan, Indian Institute of Technology, Kharagpur, India
A.K. Majumdar, Indian Institute of Technology, Kharagpur, India
SECTION V:
VIDEO FEATURE EXTRACTIONS
Chapter XI
News Video Indexing and Abstraction by Specific Visual Cues: MSC and News
Caption............................................................................................................................254
Fan Jiang, Tsinghua University, China
Yu-Jin Zhang, Tsinghua University, China
SECTION VI:
VIDEO INFORMATION RETRIEVAL
Chapter XII
An Overview of Video Information Retrieval Techniques .............................................283
Sagarmay Deb, University of Southern Queensland, Australia
Yanchun Zhang, Victoria University of Technology, Australia
Chapter XIII
A Framework for Indexing Personal Videoconference.................................................293
Jiqiang Song, Chinese University of Hong Kong, Hong Kong
Michael R. Lyu, Chinese University of Hong Kong, Hong Kong
Jenq-Neng Hwang, University of Washington, USA
Chapter XIV
Video Abstraction...........................................................................................................321
Jung Hwan Oh, The University of Texas at Arlington, USA
Quan Wen, The University of Texas at Arlington, USA
Sae Hwang, The University of Texas at Arlington, USA
Jeongkyu Lee, The University of Texas at Arlington, USA
Chapter XV
Video Summarization Based on Human Face Detection and Recognition ....................347
Hong-Mo Je, Pohang University of Science and Technology, Korea
Daijin Kim, Pohang University of Science and Technology, Korea
Sung-Yang Bang, Pohang University of Science and Technology, Korea
About the Authors..........................................................................................................379
Index ..............................................................................................................................389
vi
(cid:20)(cid:16)(cid:4)(cid:15)(cid:8)(cid:25)(cid:4)
INTRODUCTION
Video data management and information retrieval are very important areas of
research in computer technology. Plenty of research is being done in these fields at
present. These two areas are changing our lifestyles because together they cover
creation, maintenance, accessing, and retrieval of video, audio, speech, and text data
and information for video display. But still lots of important issues in these areas remain
unresolved and further research is needed to be done for better techniques and appli-
cations.
The primary objective of the book is to combine these two related areas of re-
search together and provide an up-to-date account of the work being done. We ad-
dressed research issues in those fields where some progress has already been made.
Also, we encouraged researchers, academics, and industrial technologists to provide
new and brilliant ideas on these fields that could be pursued for further research.
Section I gives an introduction. We have given general introduction of the two
areas, namely, video data management and information retrieval, from the very elemen-
tary level. We discussed the problems in these areas and some of the work done in
these fields since the last decade.
Section II defines video data storage techniques and networking. We present a
chapter that describes the design for a High-performance Data Recording Architecture
(HYDRA) that can record data in real time for large-scale servers. Although digital
continuous media (CM) is being used as an integral part of many applications and
attempts have been made for efficient retrieval of such media for many concurrent
users, not much has been done so far to implement these ideas for large-scale servers.
Then a chapter introduces video data management techniques for computational aug-
mentation of human memory, i.e., augmented memory, on wearable and ubiquitous com-
puters used in our everyday life. In another chapter, in order to organize and manipulate
vast amount of multimedia data in an efficient way, a method to summarize these digital
data has been presented. Also we present a contemporary review of the various differ-
ent strategies available to facilitate Very Low Bit-Rate (VLBR) coding for video commu-
nications over mobile and fixed transmission channels as well as the Internet.
vii
Section III talks about video data security and video data synchronization and
timeliness. We describe how to present different multimedia objects on a web-based
presentation system. A chapter is devoted to highlighting the biometrics technologies,
which are based on video sequences, viz face, eye (iris/retina), and gait.
Section IV will present various video shot boundary detection techniques. A new
robust paradigm capable of detecting scene changes on compressed MPEG video data
directly has been proposed. Then an innovative shot boundary detection method us-
ing an unsupervised segmentation algorithm and the technique of object tracking based
on the segmentation mask maps are presented. We also describe a histogram with soft
decision using the Hue, Saturation, and Intensity (HSV) color space for effective detec-
tion of video shot boundaries.
Section V will throw light on video feature extractions. We address the issues of
providing the semantic structure and generating abstraction of content in news broad-
cast.
Section VI covers video information retrieval techniques and presents an up-to-
date overview of various video information retrieval systems. As the rapid technical
advances of multimedia communication have made it possible for more and more people
to enjoy videoconferences, important issues unique to personal videoconference and a
comprehensive framework for indexing personal videoconference have been presented.
Then we have dealt with video summarization using human facial information through
face detection and recognition and also a discussion on various issues of video ab-
straction with a new approach to generate it.
The audience for this book would be researchers who are working in these two
fields. Also researchers from other areas who could start-up in these fields could find
the book useful. It could be a reference guide for researchers from other related areas as
well. Reading this book can benefit undergraduate and post-graduate students who are
interested in multimedia and video technology.
CHAPTER HIGHLIGHTS
In Chapter I, Video Data Management and Information Retrieval, we present a
basic introduction of the two very important areas of research in the domain of Informa-
tion Technology, namely, video data management and video information retrieval. Both
of these areas still need research efforts to seek solutions to many unresolved prob-
lems for efficient data management and information retrieval. We discuss those issues
and relevant work done in these two fields during the last few years.
Chapter II, HYDRA: High-performance Data Recording Architecture for Stream-
ing Media, describes the design for a High-performance Data Recording Architecture
(HYDRA). Presently, digital continuous media (CM) are well established as an integral
part of many applications. In recent years, a considerable amount of research has
focused on the efficient retrieval of such media for many concurrent users. The authors
argue that scant attention has been paid to large-scale servers that can record such
streams in real time. However, more and more devices produce direct digital output
streams either over wired or wireless networks, and various applications are emerging
to make use of them. For example, in many industrial applications, cameras now provide
the means to monitor, visualize, and diagnose events. Hence, the need arises to capture
and store these streams with an efficient data stream recorder that can handle both
viii
recording and playback of many streams simultaneously and provide a central reposi-
tory for all data. With this chapter, the authors present the design of the HYDRA
system, which uses a unified architecture that integrates multi-stream recording and
retrieval in a coherent paradigm, and hence provides support for these emerging appli-
cations.
Chapter III, Wearable and Ubiquitous Video Data Management for Computa-
tional Augmentation of Human Memory, introduces video data management techniques
for computational augmentation of human memory, i.e., augmented memory, on wear-
able and ubiquitous computers used in our everyday life. The ultimate goal of aug-
mented memory is to enable users to conduct themselves using human memories and
multimedia data seamlessly anywhere, anytime. In particular, a user’s viewpoint video
is one of the most important triggers for recalling past events that have been experi-
enced. We believe designing an augmented memory system is a practical issue for real-
world video data management. This chapter also describes a framework for an aug-
mented memory album system named Scene Augmented Remembrance Album (SARA).
In the SARA framework, we have developed three modules for retrieving, editing, trans-
porting, and exchanging augmented memory. Both the Residual Memory module and
the I’m Here! module enable a wearer to retrieve video data that he/she wants to recall
in the real world. The Ubiquitous Memories module is proposed for editing, transport-
ing, and exchanging video data via real-world objects. Lastly, we discuss future works
for the proposed framework and modules.
Chapter IV is titled Adaptive Summarization of Digital Video Data. As multime-
dia applications are rapidly spread at an ever-increasing rate, efficient and effective
methodologies for organizing and manipulating these data become a necessity. One of
the basic problems that such systems encounter is to find efficient ways to summarize
the huge amount of data involved. In this chapter, we start by defining the problem of
key frames extraction then reviewing a number of proposed techniques to accomplish
that task, showing their pros and cons. After that, we describe two adaptive algorithms
proposed in order to effectively select key frames from segmented video shots where
both apply a two-level adaptation mechanism. These algorithms constitute the second
stage of a Video Content-based Retrieval (VCR) system that has been designed at Old
Dominion University. The first adaptation level is based on the size of the input video
file, while the second level is performed on a shot-by-shot basis in order to account for
the fact that different shots have different levels of activity. Experimental results show
the efficiency and robustness of the proposed algorithms in selecting the near optimal
set of key frames required, to represent each shot.
Chapter V, Very Low Bit-rate Video Coding, presents a contemporary review of
the various different strategies available to facilitate Very Low Bit Rate (VLBR) coding
for video communications over mobile and fixed transmission channels and the Internet.
VLBR media is typically classified as having a bit rate between 8 and 64Kbps. Tech-
niques that are analyzed include Vector Quantization, various parametric model-based
representations, the Discrete Wavelet and Cosine Transforms, and fixed and arbitrary
shaped pattern-based coding. In addition to discussing the underlying theoretical prin-
ciples and relevant features of each approach, the chapter also examines their benefits
and disadvantages together with some of the major challenges that remain to be solved.
The chapter concludes by providing some judgments on the likely focus of future
research in the VLBR coding field.
ix
Chapter VI is titled Video Biometrics. Biometrics is a technology of fast, user-
friendly personal identification with a high level of accuracy. This chapter highlights
the biometrics technologies that are based on video sequences viz face, eye (iris/
retina), and gait. The basics behind the three video-based biometrics technologies are
discussed along with a brief survey.
Chapter VII is titled Video Presentation Model. Lecture-on-Demand (LOD) multi-
media presentation technologies among the network are most often used in many com-
munications services. Examples of those applications include video-on- demand, inter-
active TV, and the communication tools of a distance learning system, and so on. We
describe how to present different multimedia objects on a web-based presentation
system. Using characterization of extended media streaming technologies, we devel-
oped a comprehensive system for advanced multimedia content production: support
for recording the presentation, retrieving the content, summarizing the presentation,
and customizing the representation. This approach significantly impacts and supports
the multimedia presentation authoring processes in terms of methodology and commer-
cial aspects. Using the browser with the Windows media services allows students to
view live video of the teacher giving his speech, along with synchronized images of his
presentation slides and all the annotations/comments. In our experience, this very
approach is sufficient for use in a distance learning environment.
Chapter VIII is titled Video Shot Boundary Detection. The increasing use of
multimedia streams nowadays necessitates the development of efficient and effective
methodologies for manipulating databases storing this information. Moreover, con-
tent-based access to video data requires in its first stage to parse each video stream
into its building blocks. The video stream consists of a number of shots; each one of
them is a sequence of frames pictured using a single camera. Switching from one
camera to another indicates the transition from a shot to the next one. Therefore, the
detection of these transitions, known as scene change or shot boundary detection, is
the first step in any video analysis system. A number of proposed techniques for
solving the problem of shot boundary detection exist, but the major criticisms of them
are their inefficiency and lack of reliability. The reliability of the scene change detection
stage is a very significant requirement because it is the first stage in any video retrieval
system; thus, its performance has a direct impact on the performance of all other stages.
On the other hand, efficiency is also crucial due to the voluminous amounts of informa-
tion found in video streams.
This chapter proposes a new robust and efficient paradigm capable of detecting
scene changes on compressed MPEG video data directly. This paradigm constitutes
the first part of a Video Content-based Retrieval (VCR) system that has been designed
at Old Dominion University. Initially, an abstract representation of the compressed
video stream, known as the DC sequence, is extracted, then it is used as input to a
Neural Network Module that performs the shot boundary detection task. We have
studied experimentally the performance of the proposed paradigm and have achieved
higher shot boundary detection and lower false alarms rates compared to other tech-
niques. Moreover, the efficiency of the system outperforms other approaches by sev-
eral times. In short, the experimental results show the superior efficiency and robust-
ness of the proposed system in detecting shot boundaries and flashlights (sudden
lighting variation due to camera flash occurrences) within video shots.
Chapter IX is titled Innovative Shot Boundary Detection for Video Indexing.
Recently, multimedia information, especially the video data, has been made overwhelm-