Abstract: Principal component analysis (PCA) is a key statistical technique for multivariate data analysis. For large data sets the common approach to PCA computation is based on the standard NIPALS-PCA algorithm, which unfortunately suffers from loss of orthogonality, and therefore its applicability is usually limited to the estimation of the first few components. Here we present an algorithm based on Gram-Schmidt orthogonalization (called GS-PCA), which eliminates this shortcoming of NIPALS-PCA. Also, we discuss the GPU (Graphics Processing Unit) parallel implementation of both NIPALS-PCA and GS-PCA algorithms. The numerical results show that the GPU parallel optimized versions, based on CUBLAS (NVIDIA) are substantially faster (up to 12 times) than the CPU optimized versions based on CBLAS (GNU Scientific Library).

Abstract: An efficient approach for deriving the characteristic basis functions (CBFs) for analyzing radiation and scattering problems is proposed. The spectrum of the incident plane waves is partitioned into several angular regions that are progressively enlarged in a multilevel procedure to generate orthogonal functions. The CBFs are obtained at the final orthogonalization level, so that they cover the entire angular range. This approach to generating the CBFs is better suited for electrically large objects than the conventional approach, which utilizes the entire angular range (visible) of the plane wave spectrum in a single level.

Abstract: Least Square Methods The Least Square Algorithm Linear Least Square Methods Nonlinear Least Squares Algorithm Properties of Least Square Algorithms Examples Polynomial Approximation Gram-Schmidt Procedure of Orthogonalization Hypergeometric Function Approach to Generate Orthogonal Polynomials Discrete Variable Orthogonal Polynomials Approximation Properties of Orthogonal Polynomials Artificial Neural Networks for Input-Output Approximation Introduction Direction-Dependent Approach Directed Connectivity Graph Modified Minimal Resource Allocating Algorithm (MMRAN) Numerical Simulation Examples Multi-Resolution Approximation Methods Wavelets Bezier Spline Moving Least Squares Method Adaptive Multi-Resolution Algorithm Numerical Results Global-Local Orthogonal Polynomial MAPping (GLO-MAP) in N Dimensions Basic Ideas Approximation in 1, 2, and N Dimensions Using Weighting Functions Global-Local Orthogonal Approximation in 1-, 2-, and N-Dimensional Spaces Algorithm Implementation Properties of GLO-MAP Approximation Illustrative Engineering Applications Nonlinear System Identification Problem Statement and Background Novel System Identification Algorithm Nonlinear System Identification Algorithm Numerical Simulation Distributed Parameter Systems MLPG-Moving Least Squares Approach Partition of Unity Finite Element Method Control Distribution for Over-Actuated Systems Problem Statement and Background Control Distribution Functions Hierarchical Control Distribution Algorithm Numerical Results Appendix References Index Each chapter contains an Introduction and a Summary.

Abstract: This paper presents a fundamentally new algebraic approach to the analysis and synthesis of discrete orthogonal basis functions. It provides the theoretical background to unify Fourier, Gabor and discrete orthogonal polynomial moments. For the first time, a set of objective tests are proposed to measure the quality of basis functions. It consists of two main sections: the theoretical background on the generation and orthogonalization of basis functions together with a new solution for the computation of spectra from incomplete data, as well as the implementation of interpolation for all orthogonal basis functions; a new approach to discrete orthogonal polynomials, proving that there is one and only one unitary discrete polynomial basis. Furthermore, the concept of anisotropic moments is introduced and applied to 2D seismic data, which is an image processing problem. The new polynomial basis is numerically better conditioned than the discrete cosine transform. This opens the door to new image compression algorithms, offering a higher compression ratio than the well known JPEG method, for the same numerical effort.

Abstract: Power supply noise is becoming more and more influential on timing, though noise aware timing analysis has not been well established yet, because of several difficulties such as its dependency on input vectors and dynamic behavior. This paper proposes a static timing analysis considering power supply noise in which the dependency of noise on input vectors and spatial and temporal correlations are handled in a statistical manner. We construct a statistical model of power supply voltage that dynamically varies with spatial and temporal correlation, and represent it as a set of uncorrelated variables. We demonstrate that power voltage variation is highly correlated and adopting principal component analysis as an orthogonalization technique is effective in variable reduction. Experiments confirm the validity of our model and the accuracy of timing analysis. We also discuss the accuracy and CPU time in association with variable reduction

Abstract: We study the convergence behavior of independent component analysis (ICA) algorithms that are based on the contrast function maximization and that employ symmetric orthogonalization method to guarantee the orthogonality property of the search matrix. In particular, the characterization of the critical points of the corresponding optimization problem and the stationary points of the conventional gradient ascent and fixed point algorithms are obtained. As an interesting and a useful feature of the symmetrical orthogonalization method, we show that the use of symmetric orthogonalization enables the monotonic convergence for the fixed point ICA algorithms that are based on the convex contrast functions.

Abstract: The present invention aims to provide a user selection method which can provide a large, multiuser diversity effect with a small amount of calculation in multiuser MIMO systems, the method being a user selection method for multiuser MIMO communication, in which an orthogonal coefficient is calculated using a received SINR from a projection channel vector by using GS orthogonalization, and using the orthogonal coefficient, a correction SINR is calculated, and using this correction SINR, user selection is performed, and for a next user selection, the projection channel vector is updated, and the above processes are applied to all users.

Abstract: Cognitive radio technique has been considered as a strong solution of frequency scarcity due to the limitation of frequency resource. In this paper, we propose an orthogonal beamforming methodology which enables cognitive radio systems to coexist with primary users' systems in the same spectrum and region with no interference to the primary users' systems. The orthogonal beams are obtained using Gram-Schmidt orthogonalization based on primary users' channel state information. In addition, to increase the sum-rate of the CR systems, the proposed scheme adopts an opportunistic beamforming method. Numerical results demonstrate that the proposed scheme does not cause interference to the primary users and provides higher sum-rate capacity than that of conventional CR systems.

Abstract: The proposed null Foley-Sammon transform (NFST) method based on the Gram-Schmidt orthogonalization successfully overcomes the so-called small sample size problem with high performance in terms of recognition accuracy and low computation cost, however, the NFST method is still a linear technique in nature, so a new nonlinear feature extraction method called kernel null Foley-Sammon transform (KNFST) is presented in this paper. A major advantage of the proposed method is that it is regarded every column of the kernel matrix as a corresponding sample, which is different from other commonly used kernel-based learning algorithms. Then running NFST the in kernel matrix, nonlinear features can be extracted. Experimental results on ORL database indicate that the proposed KNFST method achieves higher recognition rate than the NFST method and other kernel-based learning algorithms.

Abstract: An orthogonalization procedure for a set of high-order hierarchical (curl)-conforming basis functions or tangential vector elements which retains the span of the Nedelec space is proposed as an alternative to the Gram-Schmidt procedure which cannot be used without compromising the Nedelec space. The resulting basis functions are compared with published basis functions in terms of conditioning and solution performance. Relatively better conditioned element matrices and improved convergence speed of an iterative solver have been observed.

Abstract: A novel projection modeling method for quantitative structure activity relationship (QSAR) and quantitative structure property relationship (QSPR) is developed in this paper. Orthogonalization of block variables is introduced to deal with the problem of variable selection. Projections based on least squares are used to construct the modeling space in order to search for the best regression directions for chemical modeling. A suitable prediction space for such a model is further defined to confine the usage range of the model. Three real data sets were analyzed to check the performance of the proposed modeling method. The results obtained from Monte-Carlo cross-validation (MCCV) showed that the proposed modeling method might provide better results for QSAR and QSPR modeling than PCR and PLS with respect to both fitting and prediction abilities. Copyright © 2007 John Wiley & Sons, Ltd.

Abstract: In this paper we introduce a new orthogonalization method. Given a real $m \times n$ matrix $A$, the new method constructs an SVD-type decomposition of the form $A = \hat U\hat\Sigma\hat V^T$. The columns of $\hat U$ and $\hat V$ are orthonormal, or nearly orthonormal, while $\hat\Sigma$ is a diagonal matrix whose diagonal entries approximate the singular values of $A$. The method has three versions: a “left-side" orthogonalization scheme in which the columns of $\hat U$ constitute an orthonormal basis of Range$(A)$, a “right-side" orthogonalization scheme in which the columns of $\hat V$ constitute an orthonormal basis of Range$(A^T)$, and a third version in which both $\hat U$ and $\hat V$ have orthonormal columns, but the decomposition is not exact. The new decompositions may substitute the SVD in many applications.

Abstract: rcsgen generates basis functions for restricted cubic splines and (optionally) their derivatives. Restricted cubic spline functions assume linearity beyond the two boundary knots. It is possible to specify knots on the original scale, as default percentiles or user specified percentiles. Orthogonalization can be peformed using Gram-Schmidt orthogonalization. When orthogonalizing a matrix is returned, which can be useful for regenerating the orthogonalized spline variables for out of sample predictions.

Abstract: The transmission apparatus (10) performs orthogonalization processing to each of the signal sequence obtained by dividing each of the more than one transmission streams into more than one signal sequence based on the orthogonality of wireless resources. The signal sequence is then sent from multiple antennas (#0 through #4). The reception apparatus (50) performs the orthogonal separation processing based on the above orthogonality to the received signals to separate multiple reception signal sequence having the orthogonal characteristic, and applies a predetermined signal separation scheme based on the propagation path estimation matrix between the reception apparatus (50) and the transmission apparatus (10) to any one of the pairs of the separated reception signal sequence, thereby separating a signal sequence before subjected to the above orthogonalization processing.

Abstract: Using channel dependent precoding, a MIMO channel can be orthogonalized, which provides throughput gains especially if a linear receiver is used. When there is a limited set of modulation and coding schemes (MCS), however, we show that orthogonalization is not always the throughput maximizing strategy. Close to the upper and lower ends of operation points of the MCS set, it may be optimal to anti-orthogonalize the channel instead.

Abstract: In this paper, we present a novel incremental algorithm for principal component analysis (PCA). The proposed algorithm is a kind of covariance-free type algorithm which requires less computation and storage space in finding out the eigenvectors, than other incremental PCA methods using a covariance matrix. The major contribution of this paper is to explicitly deal with the changing mean and to use a Gram-Schmidt Orthogonalization (GSO) for enforcing the orthogonality of the eigenvectors. As a result, more accurate eigenvectors can be found with this algorithm than other algorithms. The performance of the proposed algorithm is evaluated by experiments on the data sets with various properties and it is shown that the proposed method can find out the eigenvectors more closer to those of batch algorithm than the others.

Abstract: A Lenstra-Lenstra-Lovasz (LLL)-based technique is utilized to reduce the complexity of a MIMO detector. Basis vectors can be pre-sorted, such as by V-BLAST ordering or sorted-QR ordering, prior to applying Gram-Schmidt Orthogonalization (GSO) to further improve performance. Alternatively, a joint sorting and LLL reduction (JSAR) technique can be utilized such that after each reduction step, a vector remaining to be reduced can be selected that will minimize the overall complexity. The JSAR technique can be applied on real or complex lattice bases. LLL reduction can be stopped after a predetermined threshold is exceeded.

Abstract: This paper addresses space time convolutional code design using continuous phase modulation (CPM). The possibility of constructing full diversity space time codes is investigated. A linear modulation approximation to CPM is done. Using the Gram-Schmidt orthogonalization transform the CPM signal is generated as a vector with finite energy in a different Euclidean space. A serially concatenated CPM construction is considered in searching channel codes which are able to exploit maximum diversity. Design criteria based on the encoding scheme are derived for an arbitrary number of transmit antennas. The investigations are done for a quasi-static Rayleigh fading channel.

Abstract: A model-based approach for the decision feedback equalization of Volterra type nonlinear communication channels is proposed such that the linear model-based decision feedback equalization can be considered as a special case of the proposed approach. In designing the decision feedback equalizer, the nonlinear decision feedback equalization problem is visualized as a linear, multichannel equalization problem. A complete modified Gram–Schmidt orthogonalization of the input vector is achieved by using modified sequential processing multichannel lattice stages. The elements of the multichannel desired signal vector are then estimated from the new orthogonal set by using only scalar operations. The probability of error performance of the proposed equalizer is improved by the estimation of the elements of the desired signal vector through a sigmoid activation function so that a polynomial perceptron equalizer is realized. The comparative computational complexity calculations and performance results of the proposed decision feedback equalizer are also provided.

Abstract: This paper proposes a wideband Multiple Input - Multiple Output (MIMO) channel model and its implementation algorithm suitable for channel emulators. The model is based on a structured vector modes construction scheme, and assumes separability at space and time domains. It uses the Karhunen-Loeve Expansion (KLE) basis for the orthogonalization process both on the time delay and space domains, and is able to make channel realizations based on specific statistics with a predefined Mean Square Error (MSE).

Abstract: In this paper we propose an architecture for the development of a single input-single output (SISO) time-variant wideband channel emulator based on orthogonal functions. The proposal is adequate for hardware implementation of the stochastic wideband channel model with separable Scattering Function, allowing a big reduction on the implementation complexity when compared to direct approaches. This gain comes from the channel orthogonalization concept in which only a reduced set of artificial paths - as opposed to the large set of physical paths are implemented. This approach allows a fully characterization of the channel statistics and ensures a predefined performance in terms of the mean square error (MSE), which can not be achieved by any direct channel emulator approach.

Abstract: It has recently been proposed to orthogonalize the data streams of n interfering source-destination pairs by employing an intermediate stage of non-cooperating coherent amplify-and-forward relays. This distributed zero-forcing scheme achieves a spatial multiplexing gain n/2 and does not require multi-antenna nodes. In this paper we generalize the concept from a single to multiple relay stages. Due to the concatenation of relay stages, the gain allocation problem becomes nonlinear. We show that the zero-forcing relay gain coefficients are given by the common roots of a set of polynomials, and we identify the set of network configurations for which such roots exist. In general, there exist multiple common roots. Based on experimental results we study the impact of the root selection and the number of relay stages on achievable rates. Finally, we propose a new scheme to achieve spatial orthogonalization in a single-hop interference network. Our proposal is based on the observation, that the mathematical model of the multihop problem is identical with the one of a single-hop network if we allow signals to be bounced forth and back between sources and destinations.

Abstract: In a previous paper we presented two variants of Kovarik's approximate orthogonalization algorithm for arbitrary symmetric matrices, one with and one without explicit matrix inversion. Here we propose another inverse-free version that has the advantage of a smaller bound on the convergence factor, while the computational costs per iteration are even less than in the initial inverse-free variant. We then investigate the application of the new algorithm for the numerical solution of linear least-squares problems with a symmetric matrix. The basic idea is to modify the right-hand side of the equation during the transformation of the matrix. We prove that the sequence of vectors generated in this way converges to the minimal norm solution of the problem. Numerical tests with the collocation discretization of a first-kind integral equation demonstrate a mesh-independent behaviour and stability with respect to numerical errors introduced by the use of numerical quadrature.