Abstract: The FEAST method for solving large sparse eigenproblems is equivalent to subspace iteration with an approximate spectral projector and implicit orthogonalization. This relation allows to characterize the convergence of this method in terms of the error of a certain rational approximant to an indicator function. We propose improved rational approximants leading to FEAST variants with faster convergence, in particular, when using rational approximants based on the work of Zolotarev. Numerical experiments demonstrate the possible computational savings especially for pencils whose eigenvalues are not well separated and when the dimension of the search space is only slightly larger than the number of wanted eigenvalues. The new approach improves both convergence robustness and load balancing when FEAST runs on multiple search intervals in parallel.

Abstract: It has been previously shown that blended simultaneous-source data can be successfully separated using an iterative seislet thresholding algorithm. In this letter, I combine iterative seislet thresholding with a local orthogonalization technique via a shaping regularization framework. During the iterations, the deblended data and its blending noise section are not orthogonal to each other, indicating that the noise section contains significant coherent useful energy. Although the leakage of useful energy can be retrieved by updating the deblended data from the data misfit during many iterations, I propose to accelerate the retrieval of the leakage energy using iterative orthogonalization. It is the first time that multiple constraints are applied in an underdetermined deblending problem, and the new proposed framework can overcome the drawback of a low-dimensionality constraint in a traditional 2-D deblending problem. Simulated synthetic and field data examples show the superior performance of the proposed approach.

Abstract: To orthonormalize the columns of a dense matrix, the Cholesky QR (CholQR) requires only one global reduction between the parallel processing units and performs most of its computation using BLAS-3 kernels. As a result, compared to other orthogonalization algorithms, CholQR obtains superior performance on many of the current computer architectures, where the communication is becoming increasingly expensive compared to the arithmetic operations. This is especially true when the input matrix is tall-skinny. Unfortunately, the orthogonality error of CholQR depends quadratically on the condition number of the input matrix, and it is numerically unstable when the matrix is ill-conditioned. To enhance the stability of CholQR, we recently used mixed-precision arithmetic; the input and output matrices are in the working precision, but some of its intermediate results are accumulated in the doubled precision. In this paper, we analyze the numerical properties of this mixed-precision CholQR. Our analysis shows that ...

Abstract: We propose a novel approach to attenuate random noise based on local signal-and-noise orthogonalization. In this approach, we first remove noise using one of the conventional denoising operators, and then apply a weighting operator to the noise section in order to retrieve additional components of the useful signal. Next, the initially denoised section and the retrieved signal are combined to form the final denoised section. The weighting operator is obtained by solving a least-squares minimization problem via shaping regularization with a local-smoothness constraint. The proposed denoising approach corresponds to orthogonalizing the initially denoised signal and noise in a local manner. Field data example demonstrates an excellent performance of the proposed approach.

Abstract: Bandpass filtering is a common way to estimate ground-roll noise on land seismic data, because of the relatively low-frequency content of ground roll. However, there is usually a frequency overlap between ground roll and the desired seismic reflections that prevents bandpass filtering alone from effectively removing ground roll without also harming the desired reflections. We apply a bandpass filter with a relatively high upper bound to provide an initial imperfect separation of ground roll and reflection signal. We then apply a technique called “local orthogonalization” to improve the separation. The procedure is easily implemented, since it involves only bandpass filtering and a regularized division of the initial signal and noise estimates. We demonstrate the effectiveness of the method on an open-source set of field data.

Abstract: Nonlinear component analysis such as kernel Principle Component Analysis (KPCA) and kernel Canonical Correlation Analysis (KCCA) are widely used in machine learning, statistics and data analysis, but they cannot scale up to big datasets. Recent attempts have employed random feature approximations to convert the problem to the primal form for linear computational complexity. However, to obtain high quality solutions, the number of random features should be the same order of magnitude as the number of data points, making such approach not directly applicable to the regime with millions of data points. We propose a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD. Despite the non-convex nature of these problems, our method enjoys theoretical guarantees that it converges at the rate O(1/t) to the global optimum, even for the top k eigen subspace. Unlike many alternatives, our algorithm does not require explicit orthogonalization, which is infeasible on big datasets. We demonstrate the effectiveness and scalability of our algorithm on large scale synthetic and real world datasets.

Abstract: Hand-eye calibration that establishes the pose relationship between a robot and a laser scanner is one of the most critical tasks in robotic grinding. Calibration accuracy can significantly affect the geometry and dimensional accuracy of parts such as blades. In this paper, a hand-eye calibration approach is proposed using a standard sphere as the calibration object. More important, three valid ways to improve the calibration accuracy are given. First, based on lots of experiment data, it's found that the scanning accuracy is related to the scanning area on the sphere. Combining error synthesis theory, an appropriate scanning area is found to improve the positioning accuracy of the sphere center. Second, we proved that moving along XYZ coordinate axes are best for robot during calibration process. Third, a novel orthogonalization algorithm of the calibrated rotation matrix is presented using differential kinematics. Different from the traditional orthogonalization algorithm, the algorithm converts the multiple solutions to a optimal approximate orthogonal solution, which avoids many unstable factors. Finally, test experiments show that the calibration accuracy is up to 0.0573mm. The calibration approach can be applied in robotic grinding of nuclear blades, aviation blades and turbine blades.

Abstract: We estimate the wave speed in the acoustic wave equation from boundary measurements by constructing a reduced-order model (ROM) matching discrete time-domain data. The state-variable representation of the ROM can be equivalently viewed as a Galerkin projection onto the Krylov subspace spanned by the snapshots of the time-domain solution. The success of our algorithm hinges on the data-driven Gram--Schmidt orthogonalization of the snapshots that suppresses multiple reflections and can be viewed as a discrete form of the Marchenko--Gel'fand--Levitan--Krein algorithm. In particular, the orthogonalized snapshots are localized functions, the (squared) norms of which are essentially weighted averages of the wave speed. The centers of mass of the squared orthogonalized snapshots provide us with the grid on which we reconstruct the velocity. This grid is weakly dependent on the wave speed in traveltime coordinates, so the grid points may be approximated by the centers of mass of the analogous set of squared orthogonalized snapshots generated by a known reference velocity. We present results of inversion experiments for one- and two-dimensional synthetic models.

Abstract: Nonlinear component analysis such as kernel Principle Component Analysis (KPCA) and kernel Canonical Correlation Analysis (KCCA) are widely used in machine learning, statistics and data analysis, but they can not scale up to big datasets. Recent attempts have employed random feature approximations to convert the problem to the primal form for linear computational complexity. However, to obtain high quality solutions, the number of random features should be the same order of magnitude as the number of data points, making such approach not directly applicable to the regime with millions of data points. We propose a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD. Despite the \emph{non-convex} nature of these problems, our method enjoys theoretical guarantees that it converges at the rate $\tilde{O}(1/t)$ to the global optimum, even for the top $k$ eigen subspace. Unlike many alternatives, our algorithm does not require explicit orthogonalization, which is infeasible on big datasets. We demonstrate the effectiveness and scalability of our algorithm on large scale synthetic and real world datasets.

Abstract: In this paper, we propose algorithms which preserve energy in empirical mode decomposition (EMD), generating finite $n$ number of band limited Intrinsic Mode Functions (IMFs). In the first energy preserving EMD (EPEMD) algorithm, a signal is decomposed into linearly independent (LI), non orthogonal yet energy preserving (LINOEP) IMFs and residue (EPIMFs). It is shown that a vector in an inner product space can be represented as a sum of LI and non orthogonal vectors in such a way that Parseval's type property is satisfied. From the set of $n$ IMFs, through Gram-Schmidt orthogonalization method (GSOM), $n!$ set of orthogonal functions can be obtained. In the second algorithm, we show that if the orthogonalization process proceeds from lowest frequency IMF to highest frequency IMF, then the GSOM yields functions which preserve the properties of IMFs and the energy of a signal. With the Hilbert transform, these IMFs yield instantaneous frequencies and amplitudes as functions of time that reveal the imbedded structures of a signal. The instantaneous frequencies and square of amplitudes as functions of time produce a time-frequency-energy distribution, referred as the Hilbert spectrum, of a signal. Simulations have been carried out for the analysis of various time series and real life signals to show comparison among IMFs produced by EMD, EPEMD, ensemble EMD and multivariate EMD algorithms. Simulation results demonstrate the power of this proposed method.

Abstract: In this paper we give first results for the approximation of e A b, ie the matrix exponential times a vector, using the incomplete orthogonalization method The benefits compared to the Arnoldi iteration are clear: shorter orthogonalization lengths make the algorithm faster and a large memory saving is also possible For the case of three term orthogonalization recursions, simple error bounds are derived using the norm and the field of values of the projected operator In addition, an a posteriori error estimate is given which in numerical examples is shown to work well for the approximation In the numerical examples we particularly consider the case where the operator A arises from spatial discretization of an advection-diffusion operator

Abstract: We apply L€ canonical orthogonalization method to investigate the linearly dependent problem arising from the variational calculation of atomic systems using Slater-type orbital configuration-interaction (STO-CI) basis functions. With a specific arithmetic precision used in numerical computations, the nonorthogonal STO-CI basis is easily linearly dependent when the number of basis functions is sufficiently large. We show that L€ canonical orthogonalization method can successfully overcome such problem and simultaneously reduce the dimension of basis set. This is illustrated first through an S-wave model He atom, and then the real two-electron atoms in both the ground and excited states. In all of these calculations, the variational bound state energies of the two-electron systems are obtained in reasonably high accuracy using over-redundant STO-CI bases, however, without using extended high-precision technique. V C 2015 Wiley Periodicals, Inc.

Abstract: The block diagonalization (BD) precoding technique is a well-known linear transmit strategy for multiuser multi-input multi-output (MU-MIMO) systems. The MU-MIMO broadcast channel is decomposed into multiple independent parallel single user MIMO (SU-MIMO) channels and achieves the maximum diversity order at high data rates. The lattice reduction-aided decoding (LRAD) features the reduced decoding complexity in MIMO communications. The Lenstra-Lenstra-Lovasz (LLL) algorithm has been extensively used to obtain better bases of the channel matrix while the complex lattice reduction (CLR) is aimed at improving orthogonality of basis vectors and shortening them. The orthogonalization and size reduction work are left for the CLR algorithm so that a modification of the channel matrix is carried out, resulting in better precoding and detection performances. We also derive bounds for lattice decoding. Simulation results show that the bit error rate (BER) performance of our proposed algorithm is better than that of conventional ones and it reduces the complexity compared with the LLL algorithm-based schemes.

Abstract: The mixed-precision Cholesky QR (CholQR) can orthogonalize the columns of a dense matrix with the minimum communication cost. Moreover, its orthogonality error depends only linearly to the condition number of the input matrix. However, when the desired higher-precision is not supported by the hardware, the software-emulated arithmetics are needed, which could significantly increase its computational cost. When there are a large number of columns to be orthogonalized, this computational overhead can have a dramatic impact on the orthogonalization time, and the mixed-precision CholQR can be much slower than the standard CholQR. In this paper, we examine several block variants of the algorithm, which reduce the computational overhead associated with the software-emulated arithmetics, while maintaining the same orthogonality error bound as the mixed-precision CholQR. Our numerical and performance results on multicore CPUs with a GPU, as well as a hybrid CPU/GPU cluster, demonstrate that compared to the mixed-precision CholQR, such a block variant can obtain speedups of up to 7.1× while maintaining about the same order of the numerical errors.

Abstract: This paper proposes a general model of superdirectivity to provide analytical and closed-form solutions for arbitrary sensor arrays. Based on the equivalence between the maximum directivity factor and the maximum array gain in the isotropic noise field, Gram-Schmidt orthogonalization is introduced and recursively transformed into a matrix form to conduct pre-whitening and matching operations that result in superdirectivity solutions. A Gram-Schmidt mode-beam decomposition and synthesis method is then presented to formally implement these solutions. Illustrative examples for different arrays are provided to demonstrate the feasibility of this method, and a reduced rank technique is used to deal with the practical array design for robust beamforming and acceptable high-order superdirectivity. Experimental results that are provided for a linear array consisting of nine hydrophones show the good performance of the technique. A superdirective beampattern with a beamwidth of 48.05° in the endfire direction is typically achieved when the inter-sensor spacing is only 0.09λ (λ is the wavelength), and the directivity index is up to 12 dB, which outperforms that of the conventional delay-and-sum counterpart by 6 dB.

Abstract: The search for a canonical set of eigenvectors of the discrete Fourier transform has been ongoing for more than three decades. The goal is to find an orthogonal basis of eigenvectors which would approximate Hermite functions -- the eigenfunctions of the continuous Fourier transform. This eigenbasis should also have some degree of analytical tractability and should allow for efficient numerical computations. In this paper we provide a partial solution to these problems. First, we construct an explicit basis of (non-orthogonal) eigenvectors of the discrete Fourier transform, thus extending the results of [7]. Applying the Gramm-Schmidt orthogonalization procedure we obtain an orthogonal eigenbasis of the discrete Fourier transform. We prove that the first eight eigenvectors converge to the corresponding Hermite functions, and we conjecture that this convergence result remains true for all eigenvectors.

Abstract: It has been shown previously that blended simultaneoussource data can be successfully separated using an iterative seislet thresholding algorithm. I combine the iterative seislet thresholding with the recently proposed local signal-and-noise orthogonalization via the shaping regularization framework. During the initial iterations, the deblended data and its blending noise section are not orthogonal to each other, indicating that the noise section contains significant coherent useful energy. Although the leakage of useful energy can be retrieved by updating the deblended data from the data misfit during many iterations, I propose to accelerate the retrieval of the leakage energy using iterative orthogonalziation. Simulated synthetic and field data examples show superior performance of the proposed approach.

Abstract: Orthogonalization with the prerequisite of keeping several vectors fixed is examined. Explicit formulae are derived both for orthogonal and biorthogonal vector sets. Calculation of the inverse or square root of the entire overlap matrix is eliminated, allowing computational time reduction. In this special situation, it is found sufficient to evaluate the functions of matrices of the dimension matching the number of fixed vectors. The (bi)orthogonal sets find direct application in extending multiconfigurational perturbation theory to deal with multiple reference vectors.

Abstract: It is well known that orthogonalization of column vectors in a rectangular matrix $B$ with respect to the bilinear form induced by a nonsingular symmetric indefinite matrix $A$ can be eventually seen as its factorization $B=QR$ that is equivalent to the Cholesky-like factorization in the form $B^TAB=R^T \Omega R$, where $R$ is upper triangular and $\Omega$ is a signature matrix Under the assumption of nonzero principal minors of the matrix $M=B^T A B$ we give bounds for the conditioning of the triangular factor $R$ in terms of extremal singular values of $M$ and of only those principal submatrices of $M$ where there is a change of sign in $\Omega$ Using these results we study the numerical behavior of two types of orthogonalization schemes and we give the worst-case bounds for quantities computed in finite precision arithmetic In particular, we analyze the implementation based on the Cholesky-like factorization of $M$ and the Gram--Schmidt process with respect to the bilinear form induced by the matrix

Abstract: This paper presents an efficient architecture of a fixed-point Fast Independent Component analysis (FastICA) in field programmable gate array (FPGA). The algorithm separates up to four signals using four sensors. A prestage QR decomposition is used to improve the speed of eigenvalues and eigenvectors evaluation of the covariance matrix. Moreover, a symmetric orthogonalization of the unit estimation algorithm is implemented using an iterative technique to speed up the search algorithm for higher order data input. The algorithm is implemented using Xilinx Virtex5-XC5VLX50t FPGA. The proposed architecture can process 128 samples for the four sensors in less than 2.5 ms when the design is simulated using 100 MHz clock.

Abstract: An adaptive online algorithm with a dictionary of observed signals for kernel principal subspace analysis is presented. A coefficient matrix for eigenfunctions is updated by a recursive least squares (RLS)-type algorithm and entries in the dictionary are adaptively added / removed preserving orthogonality of the eigenfunctions. It is shown that the orthogonalization can be implemented by analytically solvable (generalized) eigenvalues of 2×2 matrices, instead of the computation of the inverse squared root of matrix having the size of the dictionary. Numerical example is then illustrated to support the analysis.

Abstract: Many learning algorithms of an artificial neural network (ANN) require the solution of linear least squares problems in order to obtain the synaptic weights. For the ANN known as Linear associator, we study learning algorithms and perform a comparative analysis from the point of view of accuracy and complexity of these algorithms. In this sense, we have proposed algorithms to solve linear least squares problem present in this network for the case of rank deficient matrices. We compare those algorithms that use the pseudoinverse considering six ways to calculate the matrix: Method of the singular value descomposition, method based on the Generalized Cholesky Factorizacition, method based in the process of Gramm-Schmidt Orthogonalization, method of the row reduce echelon form, method based on the ideas of the Gradient Projection, and the latter is a method derived from the Cayley-Hamilton theorem. Finally, we analyze, from the results of the comparative study, the behavior of the pseudoinverse in the learning process of Hopfield neural network and Radial Basis Functions.

Abstract: New theoretical background of Parlett-Kahan's "twice is enough" algorithm for computing accurate vectors in Gram-Schmidt orthogonalization is given. An unorthodox type of error analysis is applied by considering lost digits in cancellation. The resulting proof is simple and that makes it possible to calculate the number of accurate digits after all reorthogonalization steps. Self improving nature of projection matrices is found giving a possible explanation for the stability of some ABS methods. The numerical tests demonstrate the validity and applicability of the theoretical results for the CGS, MGS and rank revealing QR algorithms.

Abstract: This paper presents two novel regularization methods motivated in part by the geometric significance of biorthogonal bases in signal processing applications. These methods, in particular, draw upon the structural relevance of orthogonality and biorthogonality principles and are presented from the perspectives of signal processing, convex programming, continuation methods and nonlinear projection operators. Each method is specifically endowed with either a homotopy or tuning parameter to facilitate tradeoff analysis between accuracy and numerical stability. An example involving a basis comprised of real exponential signals illustrates the utility of the proposed methods on an ill-conditioned inverse problem and the results are compared to standard regularization techniques from the signal processing literature.

Abstract: The invention discloses a method for estimating directions of arrival of adjacent strong and weak signals for orthogonalization search. By the aid of the method, problems of weak signal loss and influence on weak signal detection probability when strong signals are suppressed by the aid of the existing methods mainly can be solved. The method includes implementation procedures of estimating data correlation matrixes by the aid of data received by array antennas; decomposing characteristic values of the correlation matrixes to obtain signal subspaces and noise subspaces; estimating the directions of arrival of the strong signals by the aid of MUSIC (multiple signal classification) spatial spectra; projecting search steering vectors in orthogonal complementary spaces of the strong signals according to matrix theories to obtain modified search steering vectors; locally searching the matched weak signals near the strong signals by the aid of the modified search steering vectors to obtain estimation of the directions of arrival of the weak signals. The method has the advantages of high weak signal detection probability and estimation precision of the directions of arrival, and can be used for estimating the directions of arrival under the conditions of adjacent strong and weak signals.

Abstract: An orthogonalization matrix calculation circuit may include a scaling coefficient calculation circuit configured to calculate a scaling coefficient for each of a plurality of candidate update operations for the orthogonalization matrix, wherein each of the plurality of candidate update operations comprises combining linearly at least one of a first column or a second column of the orthogonalization matrix previously utilized to update the orthogonalization matrix, an update operation selection circuit configured to select an optimum candidate update operation from the plurality of candidate update operations, and a matrix update circuit configured to update the orthogonalization matrix according to the scaling coefficient of the optimum candidate update operation.

Abstract: The invention discloses a method for recovering phase distribution of phase shift interference figures and a method for obtaining phase shift between the two figures. The method for obtaining the phase shift between the two figures comprises writing each interference figure in the form of a matrix and serving each interference figure as a column vector matrix; obtaining a background component through a low-pass filter; subtracting each column vector formed by the corresponding phase shift interference figure from the background component to obtain a pair of vectors and forming a subspace by the two vectors to achieve the vectorization of the interference figures, wherein an included angle is formed between the vectors; obtaining an orthogonal basis of the subspace through a basis in the subspace by a Schmidt orthogonalization method, further obtaining a corresponding standard orthogonal basis and obtaining an included angle formed between two original signals through the obtained standard orthogonal basis. According to the method for recovering the phase distribution of the phase shift interference figures and obtaining the phase shift between every two figures, the phase distribution between the two phase shift interference figures can be obtained and the changes of the phase shift can be further obtained through the series of phase shift interference figures.

Abstract: The invention discloses an emission wave beam nulling widening method based on orthogonal projection, and relates to the digital array radar emission wave beam nulling widening technology. The method utilizes a reconstruction zero setting vector matrix, and data volume is reduced greatly. In addition, in order to avoid matrix inversion operation, a Gram-Schmidt orthogonalization (GSO) thought is introduced, and a recursion method is employed to carry out deducing of orthogonal complementary space. The provided method achieves emission wave beam nulling widening rapidly and flexibly under a condition that an emission wave beam pointing direction, a zero setting direction and a nulling width are known and the anti-interference performance of the system is enhanced.