Abstract: An elementary direct proof is given for the stationarity property of Lowdin's symmetric orthogonalization scheme. © 2002 Wiley Periodicals, Inc. Int J Quantum Chem, 2002

Abstract: A novel design method for lossy Blass matrix beam-forming networks (LBNMFNs) is presented. Compared to those formerly developed, the new method allows the design of an LBMBFN in order to generate not only two simultaneous beams but also an arbitrary number of them. This skill is obtained by means of a new approach to minimize losses that allows one to transform a nonlinear multivariable programming problem into a linear one-variable problem. The solution of such a design problem, then, can be carried out in a very straightforward way by applying Gram-Schmidt orthogonalization. Such a design method takes into account also the limited availability of coupling values of directional couplers. Numerical results obtained through the application of such a design method are then presented. The ease, accuracy, and efficiency of this novel method for the design of LBMBFN make it very useful in modern applications of multibeam antenna arrays.

Abstract: Alignment has recently been proposed as a method for measuring the degree of agreement between a kernel and a learning task (Cristianini et al., 2001). Previous approaches to optimizing kernel alignment have required the eigendecomposition of the kernel matrix which can be computationally prohibitive especially for large kernel matrices. In this paper we propose a general method for optimizing alignment over a linear combination of kernels. We apply the approach to give both transductive and inductive algorithms based on the Incomplete Cholesky factorization of the kernel matrix. The Incomplete Cholesky factorization is equivalent to performing a Gram-Schmidt orthogonalization of the training points in the feature space. The alignment optimization method adapts the feature space to increase its training set alignment. Regularization is required to ensure this alignment is also retained for the test set. Both theoretical and experimental evidence is given to show that improving the alignment leads to a reduction in generalization error of standard classifiers.

Abstract: We define a natural partial order on the orthogonal group and completely describe the intervals in this partial order. The main technical ingredient is that an orthogonal transformation induces a unique orthogonal transformation on each subspace of the orthogonal complement of its fixed subspace.

Abstract: A very efficient learning algorithm for model subset selection is introduced based on a new composite cost function that simultaneously optimizes the model approximation ability and model robustness and adequacy. The derived model parameters are estimated via forward orthogonal least squares, but the model subset selection cost function includes a D-optimality design criterion that maximizes the determinant of the design matrix of the subset to ensure the model robustness, adequacy, and parsimony of the final model. The proposed approach is based on the forward orthogonal least square (OLS) algorithm, such that new D-optimality-based cost function is constructed based on the orthogonalization process to gain computational advantages and hence to maintain the inherent advantage of computational efficiency associated with the conventional forward OLS approach. Illustrative examples are included to demonstrate the effectiveness of the new approach.

Abstract: This primarily tutorial paper presents a perspective on the development of adaptive data orthogonalization (ADO) for spatial sensor array sampled data vectors. The development of ADO methods is traced from a generalization of the split array, crosscorrelation arrival angle tracker for a single source, plane wave arrival to multiple coherent arrivals. Development is followed through to the more recent maximization of the ratio of two quadratic forms that expresses beamformer array gain. Specifically, the reduced complexity implementation of a signal model error robust minimum variance distortionless response (MVDR) adaptive beamformer (ABF) is described. This robust beamformer is a simple, explicit linear blend (combination) of the inherently robust shaded time delay-and-sum conventional beamformer (CBF) and the enhanced minimum variance (EMV) dominant mode rejection (DMR) ABF. This ABF construction places an easily implemented upper bound on the signal suppression caused by signal model mismatch and allows a tradeoff between spatial resolution and main lobe signal maintenance (MLM). Several methods for robust DMR ABF are compared.

Abstract: Estimating overcomplete ICA bases for image windows is a difficult problem. Most algorithms require the estimation of values of the independent components which leads to computationally heavy procedures. Here we first review the existing methods, and then introduce two new algorithms that estimate an approximate overcomplete basis quite fast in a high-dimensional space. The first algorithm is based on the prior assumption that the basis vectors are randomly distributed in the space, and therefore close to orthogonal. The second replaces the conventional orthogonalization procedure by a transformation of the marginal density to gaussian.

Abstract: The main concern of this book is the distribution of zeros of polynomials that are orthogonal on the unit circle with respect to an indefinite weighted scalar or inner product. The first theorem of this type, proved by M. G. Krein, was a far-reaching generalization of G. Szego's result for the positive definite case. A continuous analogue of that theorem was proved by Krein and H. Langer. These results, as well as many generalizations and extensions, are thoroughly treated in this book. A unifying theme is the general problem of orthogonalization with invertible squares in modules over C*-algebras. Particular modules that are considered in detail include modules of matrices, matrix polynomials, matrix-valued functions, linear operators, and others. One of the central features of this book is the interplay between orthogonal polynomials and their generalizations on the one hand, and operator theory, especially the theory of Toeplitz marices and operators, and Fredholm and Wiener-Hopf operators, on the other hand. The book is of interest to both engineers and specialists in analysis.

Abstract: Measurement of the angular power spectrum of the cosmic microwave background is most often based on a spherical harmonic analysis of the observed temperature anisotropies. Even if all-sky maps are obtained, however, it is likely that the region around the Galactic plane will have to be removed as a result of its strong microwave emissions. The spherical harmonics are not orthogonal on the cut sky, but an orthonormal basis set can be constructed from a linear combination of the original functions. Previous implementations of this technique, based on Gram–Schmidt orthogonalization, were limited to maximum Legendre multipoles of lmax≲50, as they required all the modes have appreciable support on the cut-sky, whereas for large lmax the fraction of modes supported is equal to the fractional area of the region retained. This problem is solved by using a singular value decomposition to remove the poorly supported basis functions, although the treatment of the non-cosmological monopole and dipole modes necessarily becomes more complicated. A further difficulty is posed by computational limitations – orthogonalization for a general cut requires operations and storage and so is impractical for lmax≳200 at present. These problems are circumvented for the special case of constant (Galactic) latitude cuts, for which the storage requirements scale as and the operations count scales as . Less clear, however, is the stage of the data analysis at which the cut is best applied. As convolution is ill-defined on the incomplete sphere, beam-deconvolution should not be performed after the cut and, if all-sky component separation is as successful as simulations indicate, the Galactic plane should probably be removed immediately prior to power spectrum estimation.

Abstract: This paper presents an algorithm that adapts the parameters of a Hammerstein system model. Hammerstein systems are nonlinear systems that contain a static nonlinearity cascaded with a linear system. In this, work, the static nonlinearity is modeled using a polynomial system and the linear filter that follows the nonlinearity is an infinite impulse response system. The adaptation of the nonlinear components is enhanced in the algorithm by orthogonalizing the inputs to the coefficients of the polynomial system. The linear system is implemented as a recursive higher-order filter. The step sizes associated with the recursive components are constrained in such a way as to guarantee bounded-input, bounded-output stability of the overall system. Experimental results included in the paper show that the algorithm performs well and always converges to the global minimum if the input signal is white.

Abstract: It is well known that between all processes with independent increments, essentially only the Brownian motion and the Poisson process possess the chaotic representation property (CRP). Thus, a natural question appears: What is an appropriate analog of the CRP in the case of a general L\'evy process. At least three approaches are possible here. The first one, due to It\^o, uses the CRP of the Brownian motion and the Poisson process, as well as the representation of a L\'evy process through those processes. The second approach, due to Nualart and Schoutens, consists in representing any square-integrable random variable as a sum of multiple stochastic integrals constructed with respect to a family of orthogonalized centered power jumps processes. The third approach, never applied before to the L\'evy processes, uses the idea of orthogonalization of polynomials with respect to a probability measure defined on the dual of a nuclear space. The main aims of the present paper are to develop the three approaches in the case of a general ($\R$-valued) L\'evy process on a Riemannian manifold and (what is more important) to understand a relationship between these approaches. We apply the obtained results to the gamma, Pascal, and Meixner processes, in which case the analysis related to the orthogonalized polynomials becomes essentially simpler and richer than in the general case.

Abstract: We present a new blind source separation (BSS) approach for Bell Labs layered space-time (BLAST) communication system. The proposed algorithm is derived from the multimodulus algorithm (MMA) and employs the Gram-Schmidt orthogonalization procedure. The basic idea is to adjust the real and imaginary parts of the equalizer matrix separately and then project the updated parameters to the orthogonality constraints which ensure independence among the equalizer outputs. The simulation results show that the proposed algorithm has better performance compared to the multiuser kurtosis algorithm (MUK) with comparable computational complexities.

Abstract: We review the derivation of the fast orthogonal search algorithm, first proposed by Korenberg, with emphasis on its application to the problem of estimating coefficient matrices of vector autoregressive models. New aspects of the algorithm not previously considered are examined. One of these is the application of the algorithm to estimate coefficient matrices of a vector autoregressive process with time-varying coefficients when multiple realizations of the said process are available. Computer simulations were also performed to characterize the statistical properties of the estimates. The results show that even for shorter time series the algorithm works well and obtains good estimates of the time-varying parameters. Statistical characterization indicates that the standard deviation of the estimates decreases as \({1 \mathord{\left/ {\vphantom {1 {\sqrt N }}} \right. \kern- ulldelimiterspace} {\sqrt N }}{\text{ }}\left( N \right.\) being the length of the time series), a typical behavior of least-squares estimators. Another key aspect of the approach, which has previously been considered, is its direct extension to the parameter estimation of vector nonlinear autoregressive models. Nonlinear terms can be added to the model and the same algorithm can be applied to effectively estimate their associated parameters. Using chaotic time series generated from the Lorenz equations, the algorithm produces a model that captures the nonlinear structure of the data and exhibits the same chaotic attractor as that of the original system. © 2002 Biomedical Engineering Society.

Abstract: A subspace-projection method is developed to construct orthogonal block variable, which is originally from some kinds of series of topological indices or quantum chemical parameters. With the help of canonical correlation analysis, the orthogonal block variables were used to establish the structure-retention index correlation model. The regression of only few new orthogonal variables obtained by canonical correlation analysis against retention index shows significant improvement both in fitting and prediction ability of the correlation model. Moreover, the quantitative intercorrelation between the different block variables of topological indices can also be evaluated with the help of the subspace-projection technique proposed in this work.

Abstract: Stereophonic teleconferencing provides more natural acoustic perception by virtue of its enhanced sound localization. Of paramount importance is stereo acoustic echo cancellation (SAEC) that poses a difficult challenge to low complexity adaptive algorithms to achieve acceptable AEC due, mainly, to the strong cross-correlation between the two-channel input signals. This paper proposes a transform domain two-channel lattice algorithm that inherently decorrelates the stereo signals. The algorithm, however, bears a high computational complexity for large filter orders, N. A low complexity O(4N) algorithm is developed based on employing the functionality of the two-channel lattice cell in the previous algorithm in a weighted subband scheme. The algorithm is capable of producing complete orthogonal subbands of the stereo signals, and also allows for a tradeoff between performance and complexity. The performance of the proposed algorithms is compared with other existing algorithms via simulations and using actual teleconferencing room impulse responses.

Abstract: An analytical approach to the performance analysis of the V-BLAST algorithm is discussed in this paper. It is based on closed-form analytical models of the three key algorithm components: interference cancellation, interference nulling and optimal ordering. The closed-form analytical model of the Gram-Schmidt orthogonalization process is a key component of the proposed analysis method. It allows us to derive closed-form expressions for the signal at each detection step, to perform analytically statistical analysis for a Rayleigh-fading channel (i.e., diversity order etc.) and to obtain closed-form expressions for outage probabilities in the case of two Tx antennas. In particular, it is demonstrated that the optimal ordering does not result in increased diversity order, but only in a fixed SNR gain. Generalized versions of the V-BLAST algorithm proposed recently can be analyzed in a similar way.

Abstract: A newly introduced multivariate calibration method combining direct orthogonalization and classical least-squares is optimized and applied to the determination of the content of antioxidants and organic acids in rubber by near-infrared spectroscopy A large calibration set was built for rubbers having varying values of the target parameters, which were previously determined by reference techniques The results obtained for a validation set were compared with those rendered by partial least-squares, including several spectral pretreatment procedures

Abstract: This paper presents a new image analysis method, combining motion estimation and image segmentation. Whereas none of these methods, used on its own, delivers error free meta-information, an appropriate combination leads to an orthogonalization of these errors. This method is applied to improve the quality of motion vector fields.

Abstract: A sequential orthogonal approach to the building and training of a single hidden layer neural network is presented in this paper. The Sequential Learning Neural Network (SLNN) model proposed by Zhang and Morris [1]is used in this paper to tackle the common problem encountered by the conventional Feed Forward Neural Network (FFNN) in determining the network structure in the number of hidden layers and the number of hidden neurons in each layer. The procedure starts with a single hidden neuron and sequentially increases in the number of hidden neurons until the model error is sufficiently small. The classical Gram–Schmidt orthogonalization method is used at each step to form a set of orthogonal bases for the space spanned by output vectors of the hidden neurons. In this approach it is possible to determine the necessary number of hidden neurons required. However, for the problems investigated in this paper, one hidden neuron itself is sufficient to achieve the desired accuracy. The neural network architecture has been trained and tested on two practical civil engineering problems – soil classification, and the prediction o strength and workability of high performance concrete.

Abstract: We present, implement and test a series of incomplete orthogonal factorization methods based on Givens rotations for large sparse unsymmetric matrices. These methods include: column-Incomplete Givens Orthogonalization (cIGO-method), which drops entries by position only; column-Threshold Incomplete Givens Orthogonalization (cTIGO-method) which drops entries dynamically by both their magnitudes and positions and where the reduction via Givens rotations is done in a column-wise fashion; and, row-Threshold Incomplete Givens Orthogonalization (r-TIGO-method) which again drops entries dynamically, but only magnitude is now taken into account and reduction is performed in a row-wise fashion. We give comprehensive accounts of how one would code these algorithms using a high level language to ensure efficiency of computation and memory use. The methods are then applied to a variety of square systems and their performance as preconditioners is tested against standard incomplete LU factorization techniques. For rectangular matrices corresponding to least-squares problems, the resulting incomplete factorizations are applied as preconditioners for conjugate gradients for the system of normal equations. A comprehensive discussion about the uses, advantages and shortcomings of these preconditioners is given.

Abstract: Applications involving large sparse nonsymmetric linear systems encourage parallel implementations of robust iterative solution methods, such as GMRES(k). Two parallel versions of GMRES(k) based on different data distributions and using Householder reflections in the orthogonalization phase are analyzed with respect to scalability (their ability to maintain fixed efficiency with an increase in problem size and number of processors). A theoretical algorithm-machine model for scalability of GMRES(k) with fixed k is derived and validated by experiments on three parallel computers, each with different machine characteristics. The analysis for an adaptive version of GMRES(k), in which the restart value k is adapted to the problem, is also presented and scalability results for this case are briefly discussed.

Abstract: The Maxwell and Markov method (MNM), which also takes advantage of the knowledge of the previous solutions to extrapolate, though it does so in a way that is different from those of Altman and Mittra (see IEEE Transactions on Antennas and Propagation, vol.47, p.744 -51, 1999) is discussed. It is based on an estimation of the solution vector-to be used as an initial guess-derived by using the solutions at two or three previous frequencies as entire domain basis functions, following an application of Gram-Schmidt orthogonalization procedure. The computational time involved in generating the estimate is negligible when compared to that of the method of moments (MoM) matrix generation and the iterative solution, because the matrix equation to be solved for the purpose of estimation is very small-typically only two or three.

Abstract: In this paper, the performance of the five kinds of parallel reorthogonalization methods by using the Gram-Schmidt (G-S) method is reported. Parallelization of the re-orthogonalization process depends on the implementation of G-S orthogonalization process, i.e. Classical GS (CG-S) and Modified G-S (MG-S). To relax the parallelization problem, we propose a new hybrid method by using both the CG-S and MG-S. The HITACHI SR8000/MPP of 128 PEs, which is a distributed memory super-computer, is used in this performance evaluation.

Abstract: The complexity of an orthogonal method for solving linear systems of equations is discussed. One of the advantages of the orthogonal method is that if some equations of the initial linear system are modified, the solution of the resulting system can be easily updated with a few extra operations, if the solution of the initial linear system is used. The advantages of this procedure for this updating problem are compared with other alternative methods. Finally, a technique for reducing the condition number, based on this method, is introduced.

Abstract: A method for the orthogonalization of initially overlapping vectors, the uses for which include the re-encoding and decoding of representations triggered by sensory arrays.