Motion vector composition for video coding and transcoding

Abstract

Motion vector composition is a well-known technique for frame-skipping transcoding to accommodate bitrate fuctuation. Recently, various motion vector composition algorithms have been successfully used to facilitate fast browsing of digital video contents and encode H.264 video with the support of multiple reference frames. Fast-forward playback enables viewers to scan through the video scenes of interest in a speedy way. However, the existing video coding standards adopt motion-compensated prediction to reduce temporal redundancy of video sequences. Although it is efficient in terms of coding efficiency, this predictive scheme imposes extra constraints on how a compressed video displays since a predicted frame cannot be restored before any of its reference frames. It implies that the compressed video should be played back in a pre-determined frame order. Displaying the digital video in other orders always requires extra resources for both network traffic and decoder complexity. One approach to realize fast-forward playback is to employ a frame-skipping transcoder which transcodes only the frames required for playback at the desired fast speed. Nevertheless, it induces heavy computational complexity. Consequently, various motion vector composition (MV composition) algorithms are used to compose new motion vectors (MVs) with reduced complexity during fast-forward playback. We found that these algorithms do not work well for dropping a large number of frames, which is very common in fast-forward playback. In this thesis, a new multiple-candidate vector selection algorithm is proposed to select a composed MV from a set of candidate MVs, which utilizes relevant areas in the target macroblock to ensure a reliable tracking process for MV composition. Experimental results show that the proposed selection algorithm can provide fast-forward playback through video transcoding with significant gain, in terms of rate-distortion performance, especially when a large speed-up factor is required. In addition to fast browsing of digital video, the MV composition algorithms have been attempted to be used in multiple reference frame motion estimation (MRF-ME) for reducing the computational complexity of the encoder. However, these algorithms only perform well in a limited range of reference frames. The performance deteriorates when MV composition is processed from the current frame to a distant reference frame. In this thesis, a reliable tracking mechanism for MV composition is proposed by utilizing only the relevant areas in the target macroblock and taking different paths through a novel selection process from a set of candidate MVs. The proposed algorithm is especially suited for temporally remote reference frames in MRF-ME. Experimental results show that, compared with the existing MV composition algorithms, the proposed one can deliver a remarkable improvement on the rate-distortion performance with similar computational complexity. It is also exciting to report in this thesis that significant improvement in terms of the encoder complexity and the rate-distortion performance can be achieved when the proposed MV composition approach is applied to the hierarchical P-frame coding, which is widely adopted in the developed standards such as high-performance video coding (HVC) and multi-view video coding (MVC). We are the first to consider the approach using MV composition in these new standards. Undoubtedly, this approach could become the trend for the research in adopting the MV composition in motion estimation.