Efficient techniques for video retrieval

Abstract

Content-Based Video Retrieval (CBVR) is one of the major applications of multimedia signal analysis. Although research on this topic has been conducted for more than twenty years, many problems still remain, and better techniques for CBVR are needed. Therefore, the objectives of this thesis are to devise and develop efficient methods for video parsing and video content representation used in CBVR. In this thesis, different approaches for shot boundary detection and video content representation are reviewed. Shot boundary detection is the first step in analyzing and understanding the structure of a video for CBVR. Their accuracy will directly affect the performance of the retrieval system. However, since there are various types of transitions in a video, and the video may consist of strong motion, sudden change caused by lighting conditions, etc., the detection procedure is difficult. Moreover, video content representation plays an important role in the retrieval process because it affects the retrieval performance. Thus, efficient algorithms for CBVR remain a challenging research topic. In this research, we have proposed a robust and efficient approach based on the Colored Pattern Appearance Model (CPAM) to represent a frame for shot boundary detection. Instead of using color histogram, CPAM represents a frame by means of global statistics concerning the local visual appearance, and was originally motivated by studies in human color vision. Then, entropic thresholding is applied to determine the optimal threshold for shot boundary detection. After a video is temporally segmented into shots, a feature vector can be extracted from a shot for video retrieval based on its content. A new video content representation method has been proposed to represent a shot by considering the probability of occurrence of those pixels at the corresponding pixel position among the frames in a video shot. Experimental results show that our representation scheme outperforms the optimal key frame histogram and the alpha-trimmed average histograms. Finally, we have also developed a software library for video retrieval.