Efficient coding techniques for video with various brightness variations

Abstract

Interframe prediction is a critical component in a video encoder, where the current frame is predicted from a reference frame and only the prediction error is encoded. It assumes that the brightness keeps constant between the current and reference frames. However, some video scenes contain serious brightness variations caused by fade-in/out effects, camera iris adjustment, etc. Whenever brightness variation happens, an encoder might fail to find the true motion vectors. Consequently, it increases the number of bits to encode the prediction error. To improve coding efficiency, the H.264/AVC standard has adopted a weighted prediction (WP) tool for efficiently coding video scenes with global brightness variations (GBVs). Unfortunately, the WP tool in H.264 has not been fully examined in coding video scenes with local brightness variations (LBVs) such as illumination changes caused by a flash being fired during a press conference, a sport match, etc. Therefore, in this thesis, some novel techniques are suggested for the efficient implementation of WP in a digital video system to reduce the bitrates of encoded videos with various types of brightness variations. With the proliferation of webcams, phone cameras and video editing tools, video effects such as synthetic fading can be added to digital videos easily. The fading effects result in GBVs. Various WP models to estimate the WP parameter set have been discussed in the literature. However, no single WP model works well for diverse fading effects. In this thesis, a single reference frame multiple WP models (SRefMWP) scheme is proposed to facilitate the use of multiple WP models in a single reference frame. The proposed scheme makes a new arrangement of the frame buffers in multiple reference frame motion estimation. It enables different macroblocks in the same frame to use different WP models even when they are predicted from the same reference frame. A remarkable improvement of coding efficiency can then be achieved without modifying the H.264/AVC bitstream syntax. Afterwards, we provide a novel solution for coding scenes with flashlight. The salient characteristic of flashlight effect is the abrupt luminance change across frames of the same scene within a very short period of time, which is caused by sudden appearance of the illumination source. The proposed solution then suggests an adaptive coding order technique for increasing the efficiency of video coding by taking account of characteristics of flash scenes in video contents. The use of the adaptive coding order technique also benefits to enhance the accuracy of derived motion vectors for determination of weighting parameter sets. Coding efficiency is thus substantially improved for flash scene with different WP parameter sets applying to different MBs. Last but not least, we propose a new region-based scheme for the estimation of WP parameter sets for encoders of the H.264/AVC standard. This region-based scheme is specifically designed for handling local brightness variations in video scenes. It is achieved by making use of multiple WP parameter sets for various regions and assigning them to the same reference frame. An accurate estimation of multiple WP parameter sets is accomplished by (1) partitioning regions with a simple WP parameter estimator, (2) selecting regions where WP should be applied, and (3) estimating accurate WP parameter sets with a quasi-optimal WP parameter estimator. Similar to the SRefMWP scheme, the multiple WP parameter sets of different regions are encoded using the framework of multiple reference frames in the H.264/AVC standard. With this arrangement, the proposed scheme is compliant with the H.264/AVC standard. Results show that the region-based scheme can efficiently handle scenes with global and local brightness variations, and achieve significant coding gain over other WP schemes. By employing our proposed techniques, the work in this thesis achieves significant coding gain over the state-of-the-art WP tools. Undoubtedly, the results of our work will certainly be useful for the future development of coding videos with diverse brightness variations.