Please use this identifier to cite or link to this item:
Title: New methods for image enhancement and camera ISP learning
Authors: Liang, Zhetong
Degree: Ph.D.
Issue Date: 2021
Abstract: The captured images by modern camera sensor are color-mosaicked signals which contain incomplete color information, noise, less vivid colors and improper tones. To reconstruct a high-quality displayable image, an image signal processing (ISP) pipeline is employed onboard a camera to enhance the captured raw images by a cascade of image processing components, including demosaicking, white balance, noise removal, color space conversion, tone mapping and detail enhancement. However, there are two challenges in designing an ISP pipeline. First, the individual components in an ISP pipeline may have limited performance due to simple design. Second, there could be limitations on the whole ISP pipeline, which are designed in a divide-and-conquer manner with error accumulation. In this thesis, we leverage new optimization and learning methods to tackle the two challenges. To address the first challenge, we make several improvements on the design of individual image processing components. In the first work, we propose a new method for tone mapping component, which aims to convert a high dynamic range (HDR) image to a standard dynamic range image with improved perceptual quality. We design a hybrid l1-l0 norm optimization approach for tone mapping, and address the halo artifacts and over-enhancement problem in existing methods in the literatures. In the second work, we propose a deep-learning-based approach for single image denoising. Unlike the common end-to-end architecture, we adopt a two-stage convolutional neural network (CNN) architecture with smooth-first and enhance-later strategy. The proposed architecture removes the noise in the first stage and hallucinates high-frequency details back to the image in the second stage by adversarial learning. The proposed method can produce detail-enriched results and outperforms the existing denoising methods in terms of perceptual quality on both synthetic and real-world noisy images. In the third work, we propose a novel learning scheme for real-world burst denoising which leverages multiple images. To apply deep learning to burst denoising, it is difficult to construct a dataset for this purpose because of the object motions in a scene. We bypass this obstacle by designing a decoupled learning method to leverage two complementary datasets. With the designed network and the decoupled learning scheme, we achieve leading performance in real-world burst denoising without the need of a real-world burst dataset for training.
To address the second challenge, we propose a data-driven framework for camera ISP learning. Different from the existing camera ISPs that rely on manual design of individual image processing components, we design a deep CNN as an ISP and train it with pairwise datasets to reconstruct high-quality displayable images from raw counterparts. The challenge for this work is to properly characterize the diverse image processing components inside an ISP. We tackle this problem by designing a two-stage CNN architecture, where image restoration related subtasks are addressed in the first stage and image enhancement related subtasks in the second stage. The proposed ISP model achieves high image quality and outperforms the state-of-the-art ISP learning methods on several publicly available benchmark datasets. In summary, in this thesis, we present a novel tone mapping algorithm, and two deep CNN-based methods for image denoising and burst denoising, respectively. In addition, we present a data-driven framework for the ISP pipeline design.
Subjects: Image processing -- Digital techniques
Signal processing -- Digital techniques
Neural networks (Computer science)
Hong Kong Polytechnic University -- Dissertations
Pages: xviii, 126 pages : color illustrations
Appears in Collections:Thesis

Show full item record

Page views

Citations as of May 22, 2022

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.