Please use this identifier to cite or link to this item:
Title: Regularizing deep convolutional neural networks with a structured decorrelation constraint
Authors: Xiong, W
Du, B
Zhang, LF 
Hu, RM
Tao, DC
Keywords: Convolutional Networks
Issue Date: 2016
Publisher: Institute of Electrical and Electronics Engineers
Source: 16th IEEE International Conference on Data Mining (ICDM), Barcelona, Spain, Dec 12-15, 2016, p. 519-528 How to cite?
Abstract: Deep convolutional networks have achieved successful performance in data mining field. However, training large networks still remains a challenge, as the training data may be insufficient and the model can easily get overfitted. Hence the training process is usually combined with a model regularization. Typical regularizers include weight decay, Dropout, etc. In this paper, we propose a novel regularizer, named Structured Decorrelation Constraint (SDC), which is applied to the activations of the hidden layers to prevent overfitting and achieve better generalization. SDC impels the network to learn structured representations by grouping the hidden units and encouraging the units within the same group to have strong connections during the training procedure. Meanwhile, it forces the units in different groups to learn non-redundant representations by minimizing the cross-covariance between them. Compared with Dropout, SDC reduces the co-adaptions between the hidden units in an explicit way. Besides, we propose a novel approach called Reg-Conv that can help SDC to regularize the complex convolutional layers. Experiments on extensive datasets show that SDC significantly reduces overfitting and yields very meaningful improvements on classification performance (on CIFAR-10 6.22% accuracy promotion and on CIFAR-100 9.63% promotion).
ISBN: 978-1-5090-5473-2
ISSN: 1550-4786
DOI: 10.1109/ICDM.2016.66
Appears in Collections:Conference Paper

View full-text via PolyU eLinks SFX Query
Show full item record


Citations as of Nov 13, 2018

Page view(s)

Last Week
Last month
Citations as of Nov 18, 2018

Google ScholarTM



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.