Super-resolution videos and their application to high definition TV

Abstract

High quality video interpolation is always desirable, since the definition of video display devices is improving. It is always necessary to port a video of lower quality for display in higher quality displays, such as the conversion of SDTV videos to HDTV videos. The image/video interpolation involves the prediction of the unknown pixels from the neighboring known pixels. It is well known that using a classical linear interpolation algorithm, such as bilinear and bicubic cannot produce a visual quality that is accepted by today customer, but it has advantage on the low computational complexity. In several years before, interpolation algorithms that have low computational complexity are very important, as the processing power of the digital devices at this time is very low. However, today digital devices have a very powerful processing power. Therefore, the computational complexity is not a main concern. Nowadays, people concern more about the visual quality of the interpolated image. In order to have a high visual quality of the interpolated image, a well-known algorithm called Edge-Directed Interpolation (EDI) is proposed, which interpolate the image according to the edge direction. One of the most outstanding methods in the EDI types methods is the New Edge-Directed Interpolation (NEDI). It generates a high quality interpolated image with continuous and sharper edge. This method stimulates us to further improve it to generate the highest quality interpolated image. This thesis consists of three parts. In the first part of the thesis, we address the problems of the traditional NEDI method: prediction error and the fixed interpolation factor. These problems cause the NEDI method not suitable for the SDTV to HDTV conversion. Therefore, we develop a new eight-order sample structure for the NEDI interpolation to reduce the prediction error in the Wiener filter estimation. Moreover, we present a new fast approach for the enlargement of a SDTV video to a HDTV video with an interpolation factor of 1.5 which cannot be done by the NEDI method before. In addition, we also make an analysis on the number of the sample points used in the proposed method and its effect on regions with high frequency. In the second part of this thesis, we develop a new Adaptive Directional Window Selection method for the EDI interpolation. It can solve one of the major problems in the NEDI method, which is the covariance mismatch problem. This mismatch problem gives rise to the interpolation artifacts (prediction error) and ringing effects. As a result, the visual quality of the interpolated image is reducing. However, using our proposed method it can reduce most of the artifacts and ringing effects in the NEDI method and the performance on the high frequency and texture regions has also been improved. In the last part of this thesis, we present a pre-processing method for the Wiener filter estimation and a fast method for large up-scaling interpolation. We find that one of the reasons of the artifacts that appear in the interpolated image in NEDI method is because of the abrupt change of the pixels intensities in the sample data. Therefore, we solve this problem by using our proposed pre-processing method in the sample data before the Wiener filter estimation. This method can make the interpolation more robust to the high frequency, texture and noise region and have interpolated images with a high visual quality as compared to that of the NEDI method. Moreover, the proposed fast method can speed up the EDI interpolation when the interpolation factor is large (i.e. 4, 8, 16 times...) and keep the highest visual quality of the interpolated image.