TL;DR: This survey provides an overview of higher-order tensor decompositions, their applications, and available software.
Abstract: This survey provides an overview of higher-order tensor decompositions, their applications, and available software. A tensor is a multidimensional or $N$-way array. Decompositions of higher-order tensors (i.e., $N$-way arrays with $N \geq 3$) have applications in psycho-metrics, chemometrics, signal processing, numerical linear algebra, computer vision, numerical analysis, data mining, neuroscience, graph analysis, and elsewhere. Two particular tensor decompositions can be considered to be higher-order extensions of the matrix singular value decomposition: CANDECOMP/PARAFAC (CP) decomposes a tensor as a sum of rank-one tensors, and the Tucker decomposition is a higher-order form of principal component analysis. There are many other tensor decompositions, including INDSCAL, PARAFAC2, CANDELINC, DEDICOM, and PARATUCK2 as well as nonnegative variants of all of the above. The N-way Toolbox, Tensor Toolbox, and Multilinear Engine are examples of software packages for working with tensors.
TL;DR: This paper investigates further implications including a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length ofMatrices, angle between two matrices, and orthogonality of matrices and the use of t-linear combinations to characterize the range and kernel of a mapping defined by a third-order tensor and the t-product and the quantification of the dimensions of those sets.
Abstract: Recent work by Kilmer and Martin [Linear Algebra Appl., 435 (2011), pp. 641--658] and Braman [Linear Algebra Appl., 433 (2010), pp. 1241--1253] provides a setting in which the familiar tools of linear algebra can be extended to better understand third-order tensors. Continuing along this vein, this paper investigates further implications including (1) a bilinear operator on the matrices which is nearly an inner product and which leads to definitions for length of matrices, angle between two matrices, and orthogonality of matrices, and (2) the use of t-linear combinations to characterize the range and kernel of a mapping defined by a third-order tensor and the t-product and the quantification of the dimensions of those sets. These theoretical results lead to the study of orthogonal projections as well as an effective Gram--Schmidt process for producing an orthogonal basis of matrices. The theoretical framework also leads us to consider the notion of tensor polynomials and their relation to tensor eigentupl...
TL;DR: In this article, the phase relationship contained in the magnetotelluric (MT) impedance tensor is shown to be a second-rank tensor and the phase tensor can be depicted graphically as an ellipse, the major and minor axes representing the principal axes of the tensor.
Abstract: SUMMARY The phase relationships contained in the magnetotelluric (MT) impedance tensor are shown to be a second-rank tensor. This tensor expresses how the phase relationships change with polarization in the general case where the conductivity structure is 3-D. Where galvanic effects produced by heterogeneities in near-surface conductivity distort the regional MT response the phase tensor preserves the regional phase information. Calculation of the phase tensor requires no assumption about the dimensionality of the underlying conductivity distribution and is applicable where both the heterogeneity and regional structure are 3-D. For 1-D regional conductivity structures, the phase tensor is characterized by a single coordinate invariant phase equal to the 1-D impedance tensor phase. If the regional conductivity structure is 2-D, the phase tensor is symmetric with one of its principal axes aligned parallel to the strike axis of the regional structure. In the 2-D case, the principal values (coordinate invariants) of the phase tensor are the transverse electric and magnetic polarization phases. The orientation of the phase tensor’s principal axes can be determined directly from the impedance tensor components in both 2-D and 3-D situations. In the 3-D case, the phase tensor is nonsymmetric and has a third coordinate invariant that is a distortion-free measure of the asymmetry of the regional MT response. The phase tensor can be depicted graphically as an ellipse, the major and minor axes representing the principal axes of the tensor. 3-D model studies show that the orientations of the phase tensor principal axes reflect lateral variations (gradients) in the underlying regional conductivity structure. Maps of the phase tensor ellipses provide a method of visualizing this variation.
TL;DR: In this article, the authors consider a wide class of latent variable models, including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation, which exploit a certain tensor structure in their low-order observable moments (typically, of second and third-order).
Abstract: This work considers a computationally and statistically efficient parameter estimation method for a wide class of latent variable models--including Gaussian mixture models, hidden Markov models, and latent Dirichlet allocation--which exploits a certain tensor structure in their low-order observable moments (typically, of second- and third-order). Specifically, parameter estimation is reduced to the problem of extracting a certain (orthogonal) decomposition of a symmetric tensor derived from the moments; this decomposition can be viewed as a natural generalization of the singular value decomposition for matrices. Although tensor decompositions are generally intractable to compute, the decomposition of these specially structured tensors can be efficiently obtained by a variety of approaches, including power iterations and maximization approaches (similar to the case of matrices). A detailed analysis of a robust tensor power method is provided, establishing an analogue of Wedin's perturbation theorem for the singular vectors of matrices. This implies a robust and computationally tractable estimation approach for several popular latent variable models.