Consideration of human visual mechanism for image white balance

Abstract

The human visual system can perceive the color of an object to be relatively constant under different illumination conditions due to the mechanism of chromatic adaptation. To produce such a similar effect for images captured by cameras, white balance is used to adjust the overall color tone of the images based on the illuminant color. Various algorithms have been developed to estimate the illuminant color, with most of them focusing on the scenes under a uniform illumination, in terms of chromaticities and intensities. This dissertation aimed to investigate how to perform image white balance considering the mechanisms in the human visual system. Firstly, the effect of camera on the performance of gamut mapping white balance method was investigated using hyperspectral databases. The canonical gamut constructed using the images captured by one camera was applied to correct the images captured by the same or different camera. The canonical gamut based on an individual camera was found to have little effect on the gamut mapping algorithm performance, in terms of recovery angular errors. Then, a series of psychophysical experiments were conducted to investigate whether the white balance should always adjust images to the same white point (i.e., D65) under a uniform illumination with different chromaticities, Color Rendering Index (CRI), and illuminance, and a non-uniform illumination containing two illuminants with different combinations of chromaticities and illuminance. The observers viewed a real scene under an illumination, then adjusted the overall color tone of the scene image shown on a smartphone under a D65 illumination to match that of the scene. In general, the chromaticities and illuminance of the illumination were found to significantly affect the image white point, suggesting the necessity to appropriately adjust the image white point based on the captured scenes. Also, the findings clearly suggested an interactive effect of adapting chromaticities and illuminance on the degree of chromatic adaptation. Since white balance was seldom investigated on the images of high-dynamic range (HDR) scenes, the last part of the dissertation focused on performing white balance on HDR conditions. Various statistical white balance methods were applied on a series of LDR and fused HDR images, with recovery and reproduction angular errors used for evaluation. It was found that white balance should be applied on the fused images. A follow-up psychophysical experiment was conducted to understand how the human beings perceive a white stimulus in HDR scenes, with the stimulus luminance beyond the diffuse white luminance. The results suggested that existing uniform color spaces failed to characterize the color appearance of the stimulus and better uniform color spaces are needed for such a stimulus. In short, the dissertation helped to better understand the human perception under complicated illumination conditions and to improve the image quality of LDR and HDR images by optimizing the ISP pipeline. Future studies are needed to more comprehensively investigate the human perception under HDR scenes.