Simple Unsupervised Graph Representation Learning
Yujie Mo,Liang Peng,Jie Xu,Xiaoshuang Shi,Xiaolan Zhu +4 more
TL;DR: The proposed multiplet loss explores the complementary information between the structural information and neighbor information to enlarge the inter-class variation, as well as adds an upper bound loss to achieve the finite distance between positive embeddings and anchorembeddings for reducing the intra- class variation.
read more
Abstract: In this paper, we propose a simple unsupervised graph representation learning method to conduct effective and efficient contrastive learning. Specifically, the proposed multiplet loss explores the complementary information between the structural information and neighbor information to enlarge the inter-class variation, as well as adds an upper bound loss to achieve the finite distance between positive embeddings and anchor embeddings for reducing the intra-class variation. As a result, both enlarging inter-class variation and reducing intra-class variation result in small generalization error, thereby obtaining an effective model. Furthermore, our method removes widely used data augmentation and discriminator from previous graph contrastive learning methods, meanwhile available to output low-dimensional embeddings, leading to an efficient model. Experimental results on various real-world datasets demonstrate the effectiveness and efficiency of our method, compared to state-of-the-art methods. The source codes are released at https://github.com/YujieMo/SUGRL.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
Journal Article
Data Augmentation for Deep Graph Learning: A Survey
TL;DR: A taxonomy for graph data augmentation techniques is proposed and a structured review by categorizing the related work based on the augmented information modalities is provided, which points out promising research directions and the challenges in future research.
136
Data Augmentation for Deep Graph Learning
TL;DR: In this article , a taxonomy for graph data augmentation techniques and their applications in different deep graph learning problems is presented, and a hierarchical problem taxonomy and review of existing literature is provided.
100
Spectral Feature Augmentation for Graph Contrastive Learning and Beyond
Yifei Zhang,Hao Zhu,Zixing Song,Piotr Koniusz,Irwin King +4 more
- 02 Dec 2022
TL;DR: In this article , the spectral feature argumentation for contrastive learning on graphs and images is presented, where the authors estimate a low-rank approximation per feature map and subtract that approximation from the map to obtain its complement.
54
Multiplex Graph Representation Learning Via Dual Correlation Reduction
Yujie Mo,Yuhuan Chen,Liang Peng,Xiaoshuang Shi,Changan Yuan,Xiaolan Zhu +5 more
TL;DR: Li et al. as mentioned in this paper proposed a new framework to conduct effective and efficient self-supervised multiplex graph representation learning (SMGRL), which investigates the intra-graph and inter-graph decorrelation losses, respectively, for reducing the impact of noisy information within each graph and capturing the common information among different graphs, to achieve the effectiveness.
47
Neighbor Contrastive Learning on Learnable Graph Augmentation
TL;DR: Li et al. as mentioned in this paper proposed an end-to-end graph contrastive learning (GCL) method, named NCLA, to apply neighbor contrastive loss on learnable graph augmentation.
References
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
•Posted Content
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
TL;DR: In this article, the adaptive estimates of lower-order moments are used for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimate of lowerorder moments.
82.5K
•Posted Content
Semi-Supervised Classification with Graph Convolutional Networks
Thomas Kipf,Max Welling +1 more
TL;DR: A scalable approach for semi-supervised learning on graph-structured data that is based on an efficient variant of convolutional neural networks which operate directly on graphs which outperforms related methods by a significant margin.
22.7K
•Proceedings Article
Rectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair,Geoffrey E. Hinton +1 more
- 21 Jun 2010
TL;DR: Restricted Boltzmann machines were developed using binary stochastic hidden units that learn features that are better for object recognition on the NORB dataset and face verification on the Labeled Faces in the Wild dataset.
•Posted Content
A Simple Framework for Contrastive Learning of Visual Representations
TL;DR: It is shown that composition of data augmentations plays a critical role in defining effective predictive tasks, and introducing a learnable nonlinear transformation between the representation and the contrastive loss substantially improves the quality of the learned representations, and contrastive learning benefits from larger batch sizes and more training steps compared to supervised learning.