Error resilient support in video proxy over wireless channels

Abstract

The challenge of robust transmission is to protect compressed video data against hostile channel conditions while bringing little impact on bandwidth usage. There have been many error resilient video coding schemes proposed in the literature that attack the transmission error problem from different angles. One of the most common schemes is reference picture selection (RPS). So far, various RPS techniques have been investigated widely for use in real-time encoding. However, the RPS has not been examined in transmitting an already encoded MPEG bitstream, which have emerged as one of the indispensable video services over the Internet and 3G wireless networks nowadays. One straightforward approach in adopting the RPS scheme in an already MPEG encoded bitstream is to decode all the P-frames from the previously nearest I-frame to the current transmitted frame which is then re-encoded with a new reference; this can create undesirable complexity in the video server/proxy as well as introduce re-encoding errors. Thus, in this thesis, some novel techniques are suggested for an effective implementation of RPS in an already MPEG encoded bitstream with the minimum requirement on the server/proxy complexity. All the proposed techniques will manipulate data in the MPEG compressed domain such that the computational burden of the video server/proxy can be greatly reduced. By effectively utilizing the new compressed-domain techniques, we develop an extremely efficient structure to handle various types of macroblocks in the video streaming server. The server or proxy classifies macroblocks in the requested frame into two categories - a macroblock without motion compensation (non-MC macroblock) and a motion-compensated macroblock (MC macroblock). Two novel macroblock-based techniques are used to manipulate different types of macroblocks in the compressed domain and the server then sends the processed macroblocks to the client machine. For non-MC macroblocks, we propose to reuse motion vectors and prediction errors from the existing and already encoded MPEG bitstream in order to avoid the additional computational burden in the server/proxy. A smart quantization and dequantization pair is proposed to maintain the reconstructed quality when the RPS scheme is applied to this scenario. The server also makes use of some new compressed-domain shifting and cropping operators to handle MC-macroblocks for the purpose of further reducing its computational complexity. Experimental results show that, as compared to the RPS scheme using the conventional technique, the new approach can reduce the required server/proxy complexity significantly. It is exciting to report in this thesis that significant improvements in terms of server/proxy complexity and quality of reconstructed video can be achieved by employing our compressed-domain techniques. Undoubtedly, these techniques could become the trend for the research in adopting RPS in any already encoded MPEG video.