Please use this identifier to cite or link to this item: http://hdl.handle.net/10397/90145
PIRA download icon_1.1View/Download Full Text
Title: Towards deeper graph neural networks with differentiable group normalization
Authors: Zhou, K
Huang, X 
Li, Y
Zha, D
Chen, R
Hu, X
Issue Date: 2020
Source: In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan & H. Lin (Eds.), Advances in Neural Information Processing Systems 33 (NeurIPS 2020), p. 1-12. NeurIPS, 2020
Abstract: Graph neural networks (GNNs), which learn the representation of a node by aggregating its neighbors, have become an effective computational tool in downstream applications. Over-smoothing is one of the key issues which limit the performance of GNNs as the number of layers increases. It is because the stacked aggregators would make node representations converge to indistinguishable vectors. Several attempts have been made to tackle the issue by bringing linked node pairs close and unlinked pairs distinct. However, they often ignore the intrinsic community structures and would result in sub-optimal performance. The representations of nodes within the same community/class need be similar to facilitate the classification, while different classes are expected to be separated in embedding space. To bridge the gap, we introduce two over-smoothing metrics and a novel technique, i.e., differentiable group normalization (DGN). It normalizes nodes within the same group independently to increase their smoothness, and separates node distributions among different groups to significantly alleviate the over-smoothing issue. Experiments on real-world datasets demonstrate that DGN makes GNN models more robust to over-smoothing and achieves better performance with deeper GNNs.
Publisher: NeurIPS
Description: 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, Canada, 2020
Rights: Posted with permission of the publisher.
The following publication In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan & H. Lin (Eds.), Advances in Neural Information Processing Systems 33 (NeurIPS 2020), p. 1-12. NeurIPS, 2020 is published at NeurIPS 2020, https://proceedings.neurips.cc/paper/2020.
Appears in Collections:Conference Paper

Files in This Item:
File Description SizeFormat 
a0609-n01_580.pdf559.29 kBAdobe PDFView/Open
Open Access Information
Status open access
File Version Version of Record
Access
View full-text via PolyU eLinks SFX Query
Show full item record

Page views

104
Last Week
0
Last month
Citations as of Mar 24, 2024

Downloads

34
Citations as of Mar 24, 2024

Google ScholarTM

Check


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.