Algorithmically Transitive Network: Learning Padé Networks for Regression

TL;DR: Numerical experiments with benchmark problems show that the ATN in the form of a Pade approximant has better learning capability than linear regression analysis in a power series, the standard multi-layered neural network with the back-propagation learning, the support vector machine using the radial basis function as kernel, or the simple genetic programming.

Abstract: The learning capability of a network-based computation model named “Algorithmically Transitive Network (ATN)” is extensively studied using symbolic regression problems. To represent a variety of functions uniformly, the ATN’s topological structure is designed in the form of a truncated power series or a Pade approximant. Since the Pade approximation has better convergence properties than the Taylor expansion, the ATN with the Pade can construct an algebraic function with a relatively small number of parameters. The ATN learns with the standard back-propagation algorithm which optimizes intra-network parameters by the steepest descent method. Numerical experiments with benchmark problems show that the ATN in the form of a Pade approximant has better learning capability than linear regression analysis in a power series, the standard multi-layered neural network with the back-propagation learning, the support vector machine using the radial basis function as kernel, or the simple genetic programming.