Sprite coding techniques for video sequences

Abstract

MPEG 4 is one of the famous coding standards which suggests to use content based coding scheme. Recent researchers suggested that it is more beneficial to compression by processing global motion and local motion separately. Global motion can describe the spatial relationship among frames having overlapping background information. Therefore, it is possible to construct a single high-resolution image called sprite covering the entire visible area of the scene. Hence, a high compression ratio can be achieved. Within the encoding procedure, the core task of the sprite coding scheme is the estimation of global motion. For the estimation of global motion activities, one of the conventional approaches is to minimize the intensity discrepancy among the overlapping contents of frames. Since the minimization involves all corresponding pairs of pixels and the process has to be done in an iterative manner, the computational burden is considerably large. In view of this, a modified approach is proposed which can significantly speed up the minimization process. Note that not all the pixels taking part in the computation are beneficial to the estimation of global motion. Pixels from homogeneous area within a frame can do little in leading the intensity discrepancy value to the global minimum while they consume a significant amount of computation. Furthermore, we also have noted that pixels from homogeneous area within a frame not only cannot bring the minimization algorithm to the global minimum, but also mislead the algorithm into a wrong direction by trapping it into a local minimum. Hence, we propose to select certain representative pixels for processing which can significantly speed up the minimization process while the quality of the global motion estimation can be maintained. Experimental results show that our proposed system successfully speeds up the process by up to 6.46 times and still obtains a high quality sprite image. Other than the problem of computational complexity, the sprite coding scheme also suffers from the problem of error accumulation. There are two approaches for the accomplishment of global motion estimation in sprite generation process. The ultimate objective is to estimate a mapping function for each frame which can be used to project the frame to the space of the sprite. The first approach is to accumulate the estimated inter-frame mapping functions while the other approach is to directly estimate the mapping function between the frame and the sprite image. However, both approaches are subjected to serious problem of error accumulation. In order to alleviate this problem, we propose a robust image registration scheme by estimating the relative motion between a chosen reference frame and the current frame. Our scheme is robust against individual motion estimation error and manages to prevent the error from propagating to subsequent frames. Beside, as our proposed system does not involve the sprite image for reference purpose during the global motion estimation process, the encoder is free from blind estimation of the buffer size for the storage of the sprite. In addition, the separation of the sprite building process and the motion estimation process allows users to have various post-processing tasks to be done before the construction of the sprite. Generally speaking, sprite coding is very useful for background object compression. However, there are certain limitations to the coding scheme. Various amount of assumption has been made on the nature of the video sequence. Deviation of these assumptions might lead to degradation of the quality of the reconstructed frames. In view of this, we have proposed a scheme to update the content of the sprite. We note that there is a weak relationship between the major camera motion and the variation of frame reconstruction error. After the estimation of global motion, the encoder can insert a sprite update frame to update the sprite whenever there is a major change in camera operation. This can effectively enhance the quality of the reconstructed frames while the amount of extra bits to be sent to the decoder is very small. In short, with our proposed algorithms, the efficiency of sprite coding is greatly enhanced in terms of the computational complexity and the quality of global motion estimation. These techniques can be very useful not only in video compression field, but also in various other applications like object intrusion detection and video indexing for retrieval system.