Open AccessPosted Content
Graph Neural Ordinary Differential Equations
TL;DR: Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations.
read more
Abstract: We introduce the framework of continuous--depth graph neural networks (GNNs). Graph neural ordinary differential equations (GDEs) are formalized as the counterpart to GNNs where the input-output relationship is determined by a continuum of GNN layers, blending discrete topological structures and differential equations. The proposed framework is shown to be compatible with various static and autoregressive GNN models. Results prove general effectiveness of GDEs: in static settings they offer computational advantages by incorporating numerical methods in their forward pass; in dynamic settings, on the other hand, they are shown to improve performance by exploiting the geometry of the underlying dynamics.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Figures
Figure 1: Graph neural ordinary differential equations (GDEs) model vector fields defined on graphs, both in cases when the structure is fixed or changes in time, via a continuum of graph neural network (GNN) layers. 
Figure 4: Cora accuracy of GCDE models with different integration times s. 
Figure 5: Example position and velocity trajectories of the multi–particle system. 
Figure 8: Snapshots of the evolution of adjacency matrix At throughout the dynamics of the multi– particle system. Yellow indicates the presence of an edge and therefore a reciprocal force acting on the two bodies 
Table 3: General architecture for GCDEs on node classification tasks. GCDEs applied to different datasets share the same architecture. The vector field F is parameterized by two GCN layers. GCDEs–dopri5 shares the same structure without GDE–2 (GCN). 
Figure 6: Test extrapolation MAPE averaged across 10 experiments. Shaded area and error bars indicate 1–standard deviation intervals.
Citations
•Posted Content
Dissecting Neural ODEs
TL;DR: In this paper, a Galerkin-inspired depth-varying parameter model and neural ODEs with data-controlled vector fields are introduced, with the aim of clarifying the influence of several design choices on the underlying dynamics.
78
Coupled Graph ODE for Learning Interacting System Dynamics
Zijie Huang,Yizhou Sun,Wei Wang +2 more
- 14 Aug 2021
TL;DR: In this paper, the authors proposed coupled graph ODE, a generative model that learns the coupled dynamics of nodes and edges with a graph neural network based ODE in a continuous manner.
70
Proceedings Article
Graph-Coupled Oscillator Networks
T. K. Rusch,Benjamin Paul Chamberlain,James R. Rowbottom,Siddhartha Mishra,Michael J. Brownstein +4 more
- 04 Feb 2022
TL;DR: It is proved that GraphCON mitigates the exploding and vanishing gradients problem to facilitate training of deep multi-layer GNNs and offers competitive performance with respect to the state-of-the-art on a variety of graph-based learning tasks.
66
Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network
TL;DR: Nonlocal Kernel Network (NKN) as discussed by the authors is a nonlocal neural operator that is resolution independent, characterized by deep neural networks, and capable of handling a variety of tasks such as learning governing equations and classifying images.
Constructing Neural Network Based Models for Simulating Dynamical Systems
09 Feb 2023
TL;DR: Data-driven modeling is an alternative paradigm that seeks to learn an approximation of the dynamics of a system using observations of the true system as discussed by the authors , which can be used to solve a wide range of problems in physics and engineering.
55
References
Deep Residual Learning for Image Recognition
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016
TL;DR: In this article, the authors proposed a residual learning framework to ease the training of networks that are substantially deeper than those used previously, which won the 1st place on the ILSVRC 2015 classification task.
•Proceedings Article
Adam: A Method for Stochastic Optimization
Diederik P. Kingma,Jimmy Ba +1 more
- 01 Jan 2015
TL;DR: This work introduces Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments, and provides a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework.
138.5K
•Posted Content
Deep Residual Learning for Image Recognition
TL;DR: This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
117.9K
Long short-term memory
TL;DR: A novel, efficient, gradient based method called long short-term memory (LSTM) is introduced, which can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
99K
•Proceedings Article
Attention is All you Need
Ashish Vaswani,Noam Shazeer,Niki Parmar,Jakob Uszkoreit,Llion Jones,Aidan N. Gomez,Lukasz Kaiser,Illia Polosukhin +7 more
- 12 Jun 2017
TL;DR: This paper proposed a simple network architecture based solely on an attention mechanism, dispensing with recurrence and convolutions entirely and achieved state-of-the-art performance on English-to-French translation.
Related Papers (5)
Ricky T. Q. Chen,Yulia Rubanova,Jesse Bettencourt,David Duvenaud +3 more
- 03 Dec 2018
Emilien Dupont,Arnaud Doucet,Yee Whye Teh +2 more
- 02 Apr 2019
Sam Greydanus,Misko Dzamba,Jason Yosinski +2 more
- 04 Jun 2019
Kaiming He,Xiangyu Zhang,Shaoqing Ren,Jian Sun +3 more
- 27 Jun 2016