Abstract: Based on a general discussion of orthogonalization effects, two new one-electron orthogonalization corrections are derived to improve existing semiempirical models at the neglect of diatomic differential overlap level. The first one accounts for valence-shell orthogonalization effects on the resonance integrals, while the second one includes the dominant repulsive core–valence interactions through an effective core potential. The corrections for the resonance integrals consist of three-center terms which incorporate stereodiscriminating properties into the two-center matrix elements of the core Hamiltonian. They provide a better description of conformational properties, which is rationalized qualitatively and demonstrated through numerical calculations on small model systems. The proposed corrections are part of a new general-purpose semiempirical method which will be described elsewhere.

Abstract: We analyze the conjugate gradient (CG) method with preconditioning slightly variable from one iteration to the next. To maintain the optimal convergence properties, we consider a variant proposed by Axelsson that performs an explicit orthogonalization of the search directions vectors. For this method, which we refer to as flexible CG, we develop a theoretical analysis that shows that the convergence rate is essentially independent of the variations in the preconditioner as long as the latter are kept sufficiently small. We further discuss the real convergence rate on the basis of some heuristic arguments supported by numerical experiments. Depending on the eigenvalue distribution corresponding to the fixed reference preconditioner, several situations have to be distinguished. In some cases, the convergence is as fast with truncated versions of the algorithm or even with the standard CG method, whereas quite large variations are allowed without too much penalty. In other cases, the flexible variant effectively outperforms the standard method, while the need for truncation limits the size of the variations that can be reasonably allowed.

Abstract: Given a square matrix and single right and left starting vectors, the classical nonsymmetric Lanczos process generates two sequences of bior- thogonal basis vectors for the right and left Krylov subspaces induced by the given matrix and vectors. In this paper, we propose a Lanczos-type algorithm that extends the classical Lanczos process for single starting vectors to mul- tiple starting vectors. Given a square matrix and two blocks of right and left starting vectors, the algorithm generates two sequences of biorthogonal basis vectors for the right and left block Krylov subspaces induced by the given data. The algorithm can handle the most general case of right and left start- ing blocks of arbitrary sizes, while all previously proposed extensions of the Lanczos process are restricted to right and left starting blocks of identical sizes. Other features of our algorithm include a built-in deation procedure to detect and delete linearly dependent vectors in the block Krylov sequences, and the option to employ look-ahead to remedy the potential breakdowns that may occur in nonsymmetric Lanczos-type methods.

Abstract: The Combined Approximations (CA) method developed recently, is an effective reanalysis approach providing high quality results. In the solution process the terms of the binomial series, used as basis vectors, are first calculated by forward and back substitutions. Utilizing a Gram–Schmidt orthogonalization procedure, a new set of uncoupled basis vectors is then generated and normalized. Consequently, accurate results can be achieved by considering additional vectors, without modifying the calculations that were already carried out. In previous studies, the CA method has been used to obtain efficiently accurate approximations of the structural response in problems of linear reanalysis. It is shown in this paper that the method is most suitable for a wide range of structural optimization problems including linear reanalysis, nonlinear reanalysis and eigenvalue reanalysis. Some considerations related to the efficiency of the solution process and the accuracy of the results are discussed, and numerical examples are demonstrated. It is shown that efficient and accurate approximations are achieved for very large changes in the design.

Abstract: The V-BLAST architecture has been proposed as an extremely spectral efficient tool for wireless communications. However, owing to the intensive computation involved, it may be difficult to implement this architecture for high data rate communication system. We propose the replacement of the optimal decoding order by a suboptimal one and the utilization of the Gram-Schmitt orthogonalization (GSO) to substitute the computation of the pseudo-inverse in finding the weight vectors. These modifications significantly reduce the total number of arithmetic operations required to obtain the weight vectors and the decoding order in V-BLAST with virtually no performance degradation. In slowly time-varying fading channels, the proposed method can further reduce the amount of computation required by exploiting the time correlation between the channel gains.

Abstract: In order to come to GPS solutions of first order accuracy and integrity, carrier phase observations as well as pseudo-ranging observations have to be adjusted with respect to a linear/linearized model. Here the problem of mixed integer-real valued parameter adjustment (IRA) is met. Indeed integer cycle ambiguity unknowns have to be estimated and tested. At first we review the three concepts to deal with IRA: (i) DDD or triple difference observations are produced by a properly chosen difference operator and choice of basis, namely being free of integer-valued unknowns, (ii) The real valued unknown parameters are eliminated by a Gauss elimination step while the remaining integer-valued unknown parameters (initial cycle ambiguities) are determined by Quadratic Programming and (iii) a RA substitute model is firstly implemented (real-valued estimates of initial cycle ambiguities) and secondly a minimum distance map is designed which operates on the real-valued approximation of integers with respect to the integer data in a lattice. This is the place where the integer Gram-Schmidt or-thogonalization by means of the LLL algorithm (modified LLL algorithm) is applied being illustrated by four examples. In particular, we prove that in general it is impossible to transform an oblique base of a lattice to an orthogonal base by Gram-Schmidt orthog-onalization where its matrix entries are integer. The volume preserving Gram-Schmidt orthogonalization operator constraint to integer entries produces “almost orthogonal” bases which, in turn, can be used to produce the integer-valued unknown parameters (initial cycle ambiguities) from the LLL algorithm (modified LLL algorithm). Systematic errors generated by “almost orthogonal” lattice bases are quantified by A.K. Lenstra et al. (1992) as well as M. Pohst (1987). The solution point ẑ of Integer Least Squares generated by the LLL algorithm is ẑ = (L’)-1 [L’x] ∈ Z m where L is the lower triangular Gram-Schmidt matrix rounded to nearest integers, [L], and ẑ = [L’x] are the nearest integers of L’x, x the real valued approximation of z ∈ ℤ m , the m-dimensional lattice space Λ. Indeed due to “almost orthogonality” of the integer Gram-Schmidt procedure, the solution point ẑ is only suboptimal, only close to “least squares”.

Abstract: A BLOCK ORTHOGONALIZATION PROCEDURE WITH CONSTANT SYNCHRONIZATION REQUIREMENTS ANDREAS STATHOPOULOS AND KESHENG WU y Abstract. We propose an alternative orthonormalization method that computes the orthonor- mal basis from the right singular vectors of a matrix. Its advantage are: a all operations are matrix-matrix multiplications and thus cache-e cient, b only one synchronization point is required in parallel implementations, c could be more stable than Gram-Schmidt. In addition, we consider the problem of incremental orthonormalization where a block of vectors is orthonormalized against a previously orthonormal set of vectors and among itself. We solve this problem by alternating itera- tively between a phase of Gram-Schmidt and a phase of the new method. We provide error analysis and use it to derive bounds on how accurately the two successive orthonormalization phases should be performed to minimize total work performed. Our experiments con rm the favorable numerical behavior of the new method and its e ectiveness on modern parallel computers. Key words. Gram-Schmidt, orthogonalization, Householder, QR factorization, singular value decomposition, Poincare AMS Subject Classi cation. 65F15 1. Introduction. Computing an orthonormal basis from a given set of vectors is a basic computation, common in most scienti c applications. Often, it is also one of the most computationally demanding procedures because the vectors are of large dimension, and because the computation scales as the square of the number of vectors involved. Further, among several orthonormalization techniques the ones that ensure high accuracy are the more expensive ones. In many applications, orthonormalization occurs in an incremental fashion, where a new set of vectors we call this internal set is orthogonalized against a previously orthonormal set of vectors we call this external, and then among themselves. This computation is typical in block Krylov methods, where the Krylov basis is expanded by a block of vectors 12, 11 . It is also typical when certain external orthogonalization constraints have to be applied to the vectors of an iterative method. Locking of converged eigenvectors in eigenvalue iterative methods is such an example 19, 22 . This problem di ers from the classical QR factorization in that the external set of vectors should not be modi ed. Therefore, a two phase process is required; rst orthogonalizing the internal vectors against the external, and second the internal among themselves. Usually, the number of the internal vectors is much smaller than the external ones, and signi cantly smaller than their dimension. Another important di erence is that the accuracy of the R matrix of the QR factorization is not of pri- mary interest, but rather the orthonormality of the produced vectors Q . A variety of orthogonalization techniques exist for both phases. For the external phase, Gram- Schmidt GS and its modi ed version MGS are the most competitive choices. For the internal phase, QR factorization using Householder transformations is the most stable, albeit more expensive method 11 . When the number of vectors is signi - cantly smaller than their dimension, MGS or GS with reorthogonalization are usually preferred. Computationally, MGS, GS and Housholder transformations are based on level 1 or level 2 BLAS operations 15, 9, 8 . These basic kernel computations, dot prod- ucts, vector updates and sometimes matrix-vector operations, cannot fully utilize the Department of Computer Science, College of William and Mary, Williamsburg, Virginia 23187- 8795, andreas@cs.wm.edu . y NERSC, Lawrence Berkeley National Laboratory, Berkeley, California 94720, kwu@lbl.gov .

Abstract: A geometrical view of all known orthogonalization procedures is taken in order to understand their distinctive features and the inter-connections between them. Useful new information is gained. Its possible application to certain cognitive phenomena is also indicated.

Abstract: We give a comprehensive introduction to a general modular frame construction in Hilbert C*-modules and to related modular operators on them. The Hilbert space situation appears as a special case. The reported investigations rely on the idea of geometric dilation to standard Hilbert C*-modulesover unital C*-algebras that admit an orthonormal Riesz basis. Interrelations and applications to classical linear frame theory are indicated. As an application we describe the nature of families of operators {S_i} such that SUM_i S*_iS_i=id_H, where H is a Hilbert space. Resorting to frames in Hilbert spaces we discuss some measures for pairs of frames to be close to one another. Most of the measures are expressed in terms of norm-distances of different kinds of frame operators. In particular, the existence and uniqueness of the closest (normalized) tight frame to a given frame is investigated. For Riesz bases with certain restrictions the set of closetst tight frames often contains a multiple of its symmetric orthogonalization (i.e. L\"owdin orthogonalization).

Abstract: In earlier work we presented a set of necessary and sufficient conditions for the blind separation of a number of independent identically distributed (i.i.d.) source signals that share the same distribution and are mutually independent. We present an algorithm for implementing the multi-user Kurtosis (MUK) constrained optimization criterion suggested by these conditions. The algorithm is derived directly from the MUK cost function via a stochastic-gradient update at each iteration, followed by a Gram-Schmidt orthogonalization to project onto the criterion's constraint. A convergence analysis of the derived algorithm reveals that it is globally convergent (in the absence of noise) to a desired setting that recovers all the input sources, up to an arbitrary phase rotation each.

Abstract: A novel technique for the blind source separation (BSS) of mutually independent and identically distributed i.i.d. discrete-time sequences is presented. The observed signals are assumed mixed through a narrow-band (memoryless) multiple-input-multiple-output (MIMO) noisy channel and are then processed by a linear MIMO receiver, whose outputs should ideally match the transmitted signals. In the proposed approach (called the multi-user kurtosis (MUK) algorithm), the linear receiver's matrix setting is computed adaptively based on the optimization of a constrained statistical criterion that involves only second and fourth order statistics of the receiver's output. At each iteration, the algorithm combines a stochastic gradient adaptation with a Gram-Shmidt orthogonalization that enforces its criterion's constraints. The analysis of its stationary points, reveals that it is globally convergent to a zero forcing -ZF (or decorrelating) solution, both in the absence of noise and in the presence of spatio-temporally white additive Gaussian noise.

Abstract: An algorithm based on local scaling transformations for electronic structure calculations that scales linearly with the size of the system is presented. The key feature of the method is the absence of the orthogonalization step during iterative minimization. We illustrate the feasibility and potential of the method by applying it to total energy calculations for a variety of small clusters, viz., Na 2 , Na 7 Al, Na 20 , Si 4 , and Al 13 . The method is easily parallelizable and therefore has the potential to deal with large real life systems.

Abstract: An algebraic procedure for grid orthogonalization has been developed. It is often difficult to include both grid clustering and orthogonalization in a grid generation method. Often the degree and extent of orthogonality are hard to control when orthogonalization is included in a complicated grid generation method. Fortunately, grid orthogonalization can be performed independently of grid generation. The orthogonalization method developed is simple and includes invertibility control

Abstract: A generalization of the two-by-two rotation technique is proposed, permitting a whole row (column) of the matrix to be treated simultaneously. The method is based on the explicit analytical evaluation of the matrix exponent representing a symmetric combination of the individual rotations. Besides constructing the unitary transformation matrices, a new orthogonalization algorithm is also proposed. The resulting “unitary perturbation theory” and orthogonalization method may be useful in different areas.

Abstract: The modified Gram–Schmidt (MGS) orthogonalization process—used for example in the Arnoldi algorithm—often constitutes the bottleneck that limits parallel efficiencies. Indeed, a number of communications, proportional to the square of the problem size, are required to compute the dot-products. A block formulation is attractive but it suffers from potential numerical instability. In this paper, we address this issue and propose a simple procedure that allows the use of a block Gram—Schmidt algorithm while guaranteeing a numerical accuracy close to that of MGS. The main idea is to determine the size of the blocks dynamically. The main advantages of this dynamic procedure are two-fold: first, high performance matrix–vector multiplications can be used to decrease the execution time. Next, in a parallel environment, the number of communications is reduced. Performance comparisons with the alternative Iterated CGS also show an improvement for a moderate number of processors. Copyright © 2000 John Wiley & Sons, Ltd.

Abstract: In this paper we analyze implementations of parallel Gram-Schmidt orthogonalization algorithms. One of the first parallel orthogonalization of Gram-Schmidt was the row-wise partitioning of O'Leary and Whitman. In this paper we describe a pipelined implementation which uses column-wise partitioning schemes. Timing models for the column-wise parallel algorithms are derived. We compare our column-wise partitionings against the row-wise partitioning and validate our study with computational results. The pipelined orthogonalization algorithm is important because the timing analysis is independent of the architecture model. Threshold values of m max, which is the number of rows where row partitioning becomes better than column partitioning are found theoretically and verified with our experiments

Abstract: Because the number of parameters required by a Volterra series grows rapidly with both the length of its memory and the order of its nonlinearity, methods for identifying these models from measurements of input/output data are limited to low-order systems with relatively short memories. To deal with these computational and storage requirements one can either make extensive use of the structure of the Volterra series estimation problem to eliminate redundant storage and computations (e.g., the fast orthogonal algorithm), or apply a basis expansion, such as a Laguerre expansion, which seeks to reduce the number of model parameters, and hence, the size of the estimation problem. The use of an appropriate expansion basis can also decrease the noise sensitivity of the estimates. In this paper, we show how fast orthogonalization techniques can be combined with an expansion onto an arbitrary basis. We further demonstrate that the orthogonalization and expansion may be performed independently of each other. Thus, the results from a single application of the fast orthogonal algorithm can be used to generate multiple basis expansions. Simulations, using a simple nonlinear model of peripheral auditory processing, show the equivalence between the kernels estimated using a direct basis expansion, and those computed using the fast, implicit basis expansion technique which we propose. Running times for this new algorithm are compared to those for existing techniques. © 2000 Biomedical Engineering Society.

Abstract: In this paper, a new precoding scheme is designed allowing the complete orthogonalization of the users in the uplink of a multiuser CDMA system based on burst transmission. With this channel adapted precoding, all the inter-symbol and inter-user interferences are eliminated thanks to the appropriate linear processing at the receiver. If the received power is fixed, this system minimizes the variance of the symbol estimation error. An infinity of solutions exists to orthogonalize the system. Two objectives are added: a possible recursion in the number of users and a minimization of the average emitted power. The last optimization problem is a complex one. An approximate solution is proposed. It is shown that the system introduced in this paper outperforms burst system using a conventional set of codes.

Abstract: This paper presents a sub-voxel analysis method for multi- parameter volumetric images such as MRI to provide partial volume estimation. The proposed method finds a continuous function for a neighboring structure of each voxel. The estimation function and neighboring structure are chosen from the quadratic/cubic polynomials and a set of 2D/3D symmetric neighborhood architectures, respectively. Then, a new form of the eigenimage method, based on Gram-Schmidt orthogonalization, is derived for each choice of estimation function and neighboring structure. Finally, the above estimators are applied to a simulation model consisting of materials similar to CSF, WM, and GM of the human brain in T1- , T2-, and PD-weighted MRI. In the presence of noise, the examined continuous estimators show a smaller standard deviation (up to 40%) than the standard eigenimage method. Also the chosen estimators have analytical solution for their Gram-Schmidt filters, so their execution times are comparable with that of the standard eigenimage method. In addition, the proposed approach can determine the 3D distribution of each material and extract the connecting surfaces of the materials within each voxel.© (2000) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.

Abstract: As is known, Zernike polynomials find broad application for the solution of many problems of computational optics. The well-known Zernike polynomials are particularly attractive for their unique properties over a circular aperture. Zernike circle polynomials are used for describing both classical aberrations in optical system and aberrations related to atmospheric turbulence. There are several numerical techniques to solve for the value of Zernike coefficients, the least-squares matrix inversion method and the Gram-Schmidt orthogonalization method would become ill- conditioned due to an improper data sampling. In this article, we present the 2D discrete wavelet transform (DWT) technique to find the 3rd order spherical and coma aberration coefficients. The method offers great improvement in the accuracy and calculating speed of the fitting aberration coefficients better than the least-squares matrix inversion method and the Gram-Schmidt orthogonalization method. Furthermore, the result of solving coefficients through the 2D DWT is independent of the order of the polynomial expansion. So we can find an accurate value from the datum of fitting.

Abstract: Our work under this support broadly falls into five categories: automatic differentiation, sparsity, constraints, parallel computation, and applications. Automatic Differentiation (AD): We developed strong practical methods for computing sparse Jacobian and Hessian matrices which arise frequently in large scale optimization problems [10,35]. In addition, we developed a novel view of "structure" in applied problems along with AD techniques that allowed for the efficient application of sparse AD techniques to dense, but structured, problems. Our AD work included development of freely available MATLAB AD software. Sparsity: We developed new effective and practical techniques for exploiting sparsity when solving a variety of optimization problems. These problems include: bound constrained problems, robust regression problems, the null space problem, and sparse orthogonal factorization. Our sparsity work included development of freely available and published software [38,39]. Constraints: Effectively handling constraints in large scale optimization remains a challenge. We developed a number of new approaches to constrained problems with emphasis on trust region methodologies. Parallel Computation: Our work included the development of specifically parallel techniques for the linear algebra tasks underpinning optimization algorithms. Our work contributed to the nonlinear least-squares problem, nonlinear equations, triangular systems, orthogonalization, and linear programming. Applications: Our optimization work is broadly applicablemore » across numerous application domains. Nevertheless we have specifically worked in several application areas including molecular conformation, molecular energy minimization, computational finance, and bone remodeling.« less

Abstract: The single minor component extraction algorithm proposed by Douglas et al. is extended to a multiple minor components extraction algorithm by combining the deflation technique and the Gram-Schmidt orthogonalization. The second order analysis for the multiple case is presented by applying the averaging method or the ordinary differential equation (ODE) method. The error covariances of the estimated minor components are derived and the validity of these evaluations is demonstrated by simulations.

Abstract: Arnoldi's method and the Incomplete Orthogonalization Method (IOM) for large non-Hermitian linear systems are studied. It is shown that the inverse of a general nonsingular j × j Hessenberg matrix can be updated in O ( j 2 ) flops from that of its ( j − 1) × ( j − 1) principal submatrix. The updating recursion of inverses of the Hessenberg matrices does not need any QR or LU decomposition as commonly used in the literature. Some updating recursions of the residual norms and the approximate solutions obtained by these two methods are derived. These results are appealing because they allow one to decide when the methods converge and show one how to compute approximate solutions very cheaply and easily.

Abstract: In this paper we present a new method to estimate the radio mobile channel. The used technique is based on pilot symbols. Pilot symbols are introduced among the data frame. At reception the pilot symbols are projected on an orthonormal basis of polynomials which we call Schmidt polynomials (they are obtained with orthogonalization of Schmidt). Then an interpolation is done which gives the channel coefficient at each position in the frame. Compared to the classical method of channel estimation (based on the FFT) this method presents the same performance with less complexity.

Abstract: PROBLEM TO BE SOLVED: To obtain an adaptation controlling method for a periodicity signal capable of improving an adaptation speed and converging an error signal in a shorter time. SOLUTION: This adaptation controlling method for a periodicity signal has an adaptation signal generation algorithm 11 that is subjected to orthogonalization representation, an adaptation coefficient vector update algorithm 12 and a transfer characteristic estimation algorithm 13. The algorithm 12 adaptively adjusts orthogonalization coefficients p (n) and q (n) of an adaptation signal y (n). Necessary estimation orthogonalization coefficients P'(n) and Q'(n) are supplied from the algorithm 13. Since the coefficients p (n) and q (n) and the coefficients P'(n) and Q'(n) are variables of the same ranks respectively, one convergence does not obstruct the other any more. Consequently an adaptation speed is improved more and an error signal e (n) can be converged in a shorter time.