Please use this identifier to cite or link to this item:
PIRA download icon_1.1View/Download Full Text
DC FieldValueLanguage
dc.contributorDepartment of Applied Mathematicsen_US
dc.creatorGuo, Xen_US
dc.creatorHu, Ten_US
dc.creatorWu, Qen_US
dc.publisherMIT Pressen_US
dc.rights©2020 Xin Guo, Ting Hu and Qiang Wu.en_US
dc.rightsLicense: CC-BY 4.0, see Attribution requirements are provided at
dc.rightsThe following publication Guo, X., Hu, T., & Wu, Q. (2020). Distributed minimum error entropy algorithms. Journal of Machine Learning Research, 21(126), 1-31 is available at
dc.titleDistributed minimum error entropy algorithmsen_US
dc.typeJournal/Magazine Articleen_US
dcterms.abstractMinimum Error Entropy (MEE) principle is an important approach in Information Theoretical Learning (ITL). It is widely applied and studied in various fields for its robustness to noise. In this paper, we study a reproducing kernel-based distributed MEE algorithm, DMEE, which is designed to work with both fully supervised data and semi-supervised data. The divide-and-conquer approach is employed, so there is no inter-node communication overhead. Similar as other distributed algorithms, DMEE significantly reduces the computational complexity and memory requirement on single computing nodes. With fully supervised data, our proved learning rates equal the minimax optimal learning rates of the classical pointwise kernel-based regressions. Under the semi-supervised learning scenarios, we show that DMEE exploits unlabeled data effectively, in the sense that first, under the settings with weak regularity assumptions, additional unlabeled data significantly improves the learning rates of DMEE. Second, with sufficient unlabeled data, labeled data can be distributed to many more computing nodes, that each node takes only O(1) labels, without spoiling the learning rates in terms of the number of labels. This conclusion overcomes the saturation phenomenon in unlabeled data size. It parallels a recent results for regularized least squares (Lin and Zhou, 2018), and suggests that an inflation of unlabeled data is a solution to the MEE learning problems with decentralized data source for the concerns of privacy protection. Our work refers to pairwise learning and non-convex loss. The theoretical analysis is achieved by distributed U-statistics and error decomposition techniques in integral operators.en_US
dcterms.accessRightsopen accessen_US
dcterms.bibliographicCitationJournal of machine learning research, 2020, v. 21, no. 126, p. 1-31en_US
dcterms.isPartOfJournal of machine learning researchen_US
dc.description.validate202009 bcrcen_US
dc.description.oaVersion of Recorden_US
Appears in Collections:Journal/Magazine Article
Files in This Item:
File Description SizeFormat 
Distributed_Minimum_Error.pdf470.68 kBAdobe PDFView/Open
Open Access Information
Status open access
File Version Version of Record
View full-text via PolyU eLinks SFX Query
Show simple item record

Page views

Last Week
Last month
Citations as of Sep 24, 2023


Citations as of Sep 24, 2023

Google ScholarTM


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.