Auxiliary supervision for regularizing deep learning based image classification

Abstract

Image classification is a fundamental task in visual recognition. Deep learning-based methods, i.e., Deep Neural Networks (DNNs), are state-of-the-art approach that achieves remarkable performance. Besides, DNNs pre-trained on image classification tasks with large-scale datasets show excellent transferability for solving downstream tasks, such as semantic segmentation, object detection, etc. Therefore, image classification becomes one of the fundamental but critical tasks in visual recognition. However, DNNs easily overfit and are hard to optimize, as they have billions or millions of parameters. To tackle this challenge, regularization techniques such as data augmentations and auxiliary learning are introduced to auxiliary supervise DNNs to achieve better generalization and robustness. In this thesis, we first review existing regularization techniques in terms of data augmentation and auxiliary learning. Then we conduct two research works for regularizing DNNs on the classification task. More specifically, in the first work, we study the problem of computational color naming (CCN). We explore utilizing domain knowledge of the RGB Color Model as auxiliary supervision to regularize the model. Based on this, we expand CCN’s application to data augmentation by designing a new data augmentation method named Partial Color Jittering(PCJ). PCJ performs the color jittering on a subset of pixels of the same image color, which significantly increases images’ diversity, thereby consistently improving image classification performance. In the second work, we study the problem in vision loss estimation. We first explore that vanilla models easily overfit and fall into trivial solutions in vision loss estimation. To tackle this challenge, we propose a novel method for vision loss estimation. In detail, we formulate VF estimation as an ordinal classification problem, following the ordinal properties of the studied data. Besides, we introduce an auxiliary task to assist the generalization of the model, where the auxiliary task explicitly regularizes the model. Finally, we conclude this thesis, discuss the open challenges and address future directions.