TL;DR: The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm, and the improved memory and time efficiencies are especially true for large sized patterns training.
Abstract: The improved computation presented in this paper is aimed to optimize the neural networks learning process using Levenberg-Marquardt (LM) algorithm. Quasi-Hessian matrix and gradient vector are computed directly, without Jacobian matrix multiplication and storage. The memory limitation problem for LM training is solved. Considering the symmetry of quasi-Hessian matrix, only elements in its upper/lower triangular array need to be calculated. Therefore, training speed is improved significantly, not only because of the smaller array stored in memory, but also the reduced operations in quasi-Hessian matrix calculation. The improved memory and time efficiencies are especially true for large sized patterns training.
TL;DR: In this paper, a unified concept of using systolic arrays to perform real-time triangularization for both general and band matrices is presented, and a framework is presented for the solution of linear systems with pivoting and for least squares computations.
TL;DR: By following the formulation of elimination as a combinatorial process a considerable insight into the elimination process by studying the evolution of the cycle structure and the vertex-separator, or cut-set, structure of a graph under elimination can be gained.
Abstract: Publisher Summary This chapter provides an overview of a graph-theoretic study of the numerical solution of sparse positive definite systems of linear equations. By following the formulation of elimination as a combinatorial process a considerable insight into the elimination process by studying the evolution of the cycle structure and the vertex-separator, or cut-set, structure of a graph under elimination can be gained. Furthermore, by counting the arithmetic operations necessary to effect the decompositions, these criteria for optimization is related to the computational complexity of calculations involving the elimination process. A graph-theoretic approach for dealing with sparse systems in regard to Gaussian elimination is to attempt to find permutation matrices P, Q such that A = PMQ, which is block lower triangular, as in this case it is necessary only to decompose the diagonal blocks of PMQ. Naturally, such a transformation does not preserve symmetry. However, results of these are not applicable when M is symmetric positive definite and irreducible, because the algorithm would then produce only one diagonal block, M.
TL;DR: In this article, a modification of Henderson's procedure for finding the diagonal elements of an L (or A) matrix which does not require that L or A be stored in memory is described.
Abstract: A numerator relationship matrix for a group of animals is, by definition, the matrix with the ijth off-diagonal element equal to the numerator of Wright's [1922] coefficient of relationship between the ith and jth animals and with the ith diagonal element equal to 1 + fi where fi is Wright's [1922] coefficient of inbreeding for the ith animal. The numerator relationship matrix, say A, can be computed recursively (see Emik and Terrill [1949]), and for most situations, inbreeding and relationship coefficients can be calculated with a computer more rapidly in this manner than by path coefficient methods (Wright [1922]). The exception to this is when the dimension of A is too large for it to be stored in computer memory. Then computation of A is exceedingly time consuming. In addition to its usefulness for obtaining inbreeding and relationship coefficients, the inverse of A is required for best linear unbiased prediction of breeding values (Henderson [1973]) but, in general, A is too large to invert by conventional means. Recently, however, Henderson [1976] has described methods for computing a lower triangular matrix Z, defined such that LL' = A, with the object of computing A` = (L')1(LX1. He discovered that A-1 can be found directly from a list of sires and dams and the diagonal elements of L. Since the latter are functions of the diagonal elements of A, A1 for a noninbred population can be computed without having to compute either A or L. However, for an inbred population, the diagonal elements of L (or A) must first be found and when L is too large to store in computer memory, this can be very time consuming if Henderson's computing formulas are used. The purpose of this paper is to describe a modification of Henderson's procedure for finding the diagonal elements of an L (or A) matrix which does not require that L (or A) be stored in memory. It is therefore possible to compute rapidly inbreeding coefficients or the inverse of a numerator relationship matrix for very large numbers of animals. For example, less than three minutes were required by an IBM 370/135 to compute the diagonal elements and the inverse of a numerator relationship matrix for 1000 animals. Use of this procedure
TL;DR: In this article, a complete abstract invariant and a set of canonical forms under dynamic compensation for linear systems characterized by proper, rational transfer matrices are presented. But the complexity of the problem is not addressed.
Abstract: This paper is concerned with the development of a complete abstract invariant as well as a set of canonical forms under dynamic compensation for linear systems characterized by proper, rational transfer matrices. More specifically, it is shown that one can always associate with any proper rational transfer matrix, $T(s)$, a special lower left triangular matrix, $\xi _T (s)$, called the interactor. This matrix is then shown to represent an abstract invariant under dynamic compensation which, together with the rank of $T(s)$, represents a complete abstract invariant. A set of canonical forms under dynamic compensation is also developed along with appropriate dynamic compensation.