Abstract: This paper proposes a new approach for the evaluation of the conventional modes, i.e. rigid, distortional, local and Bredt shear-modes, to be used in the framework of the Generalised Beam Theory (GBT) for the analysis of thin-walled members. The new method identifies a set of conventional modes in a single step cross-sectional analysis and for any type of cross-section, i.e. open, closed and partially-closed ones. The algorithm differs from that of the classical GBT, which requires a two-step evaluation procedure, consisting of an initial choice of the vector basis and its successive orthogonalization. The method is based on a definition of a new quadratic functional, whose steady condition leads to an eigenvalue problem, and directly generates the sought orthogonal basis, here found using a finite-element approach. The accuracy of the proposed method is validated by means of two numerical examples, one dealing with a lipped C-section and one with a partially-closed profile. It is shown that the conventional modes derived with the proposed approach are identical to those determined with the classical two-step procedure, thus limiting the computational effort required in their identification.

Abstract: This paper presents a branch-and-lift algorithm for solving optimal control problems with smooth nonlinear dynamics and potentially nonconvex objective and constraint functionals to guaranteed global optimality This algorithm features a direct sequential method and builds upon a generic, spatial branch-and-bound algorithm A new operation, called lifting, is introduced, which refines the control parameterization via a Gram---Schmidt orthogonalization process, while simultaneously eliminating control subregions that are either infeasible or that provably cannot contain any global optima Conditions are given under which the image of the control parameterization error in the state space contracts exponentially as the parameterization order is increased, thereby making the lifting operation efficient A computational technique based on ellipsoidal calculus is also developed that satisfies these conditions The practical applicability of branch-and-lift is illustrated in a numerical example

Abstract: The Generalized Minimum Residual (GMRES) method is a popular Krylov subspace projection method for solving a nonsymmetric linear system of equations. On modern computers, communication is becoming increasingly expensive compared to arithmetic operations, and a communication-avoiding variant (CA-GMRES) may improve the performance of GMRES. To further enhance the performance of CA-GMRES, in this paper, we propose two techniques, focusing on the two main computational kernels of CA-GMRES, tall-skinny QR (TSQR) and matrix powers kernel (MPK). First, to improve the numerical stability of TSQR based on the Cholesky QR (CholQR) factorization, we use higher-precision arithmetic at carefully-selected steps of the factorization. In particular, our mixed-precision CholQR takes the input matrix in the standard \(64\)-bit double precision, but accumulates some of its intermediate results in a software-emulated double-double precision. Compared with the standard CholQR, this mixed-precision CholQR requires about \(8.5\times \) more computation but a much smaller increase in communication. Since the computation is becoming less expensive compared to the communication on a newer computer, the relative overhead of the mixed-precision CholQR is decreasing. Our case studies on a GPU demonstrate that using higher-precision arithmetic for this small but critical segment of the algorithm can improve not only the overall numerical stability of CA-GMRES but also, in some cases, its performance. We then study an adaptive scheme to dynamically adjust the step size of MPK based on the static inputs and the performance measurements gathered during the first restart loop of CA-GMRES. Since the optimal step size of MPK is often much smaller than that of the orthogonalization kernel, the overall performance of CA-GMRES can be improved using different step sizes for these two kernels. Our performance results on multiple GPUs show that our adaptive scheme can choose a near optimal step size for MPK, reducing the total solution time of CA-GMRES.

Abstract: One of the main objectives of watermarking is to achieve a better tradeoff between robustness and high visual quality of a host image. In recent years, there has been a significant development in gray-level image watermarking using fractal-based method. This paper presents a human visual system (HVS) based fractal watermarking method for color images. In the proposed method, a color pixel is considered as a 3-D vector in RGB space. And a general form of 3 × 3 matrix is utilized as the scaling operator. Meanwhile, the luminance offset vector is substituted by the range block mean vector. Then an orthogonalization fractal color coding method is achieved to obtain very high image quality. We also show that the orthogonalization fractal color decoding is a mean vector-invariant iteration. So, the range block mean vector is a good place for hiding watermark. Furthermore, for consistency with the characteristics of the HVS, we carry out the embedding process in the CIE  space and incorporate a just noticeable difference (JND) profile to ensure the watermark invisibility. Experimental results show that the proposed method has good robustness against various typical attacks, at the same time, with an imperceptible change in image quality.

Abstract: To improve the stability of the traditional natural gradient independent component analysis (ICA) algorithm and the accuracy of its separated results, a adaptive step-size natural gradient ICA algorithm with weighted orthogonalization is proposed. First, to take advantage of the pre-whitening pre-processing and keep the equivariance property of the ICA algorithm, based on the weighted orthogonal constraint on the separating matrix without pre-whitening of observed signals, weighted orthogonalization is introduced after the traditional gradient update. Then, according to the error estimation from the smoothed distance between separated outputs and optimal outputs, we obtain two adaptive step sizes based, respectively, on an unconstrained natural gradient ICA process and a weighted orthogonalization process. Simulation experiment results show that the speed of convergence of the adaptive step-size natural gradient ICA algorithms with weighted orthogonalization are faster than the traditional one; also, the stability of the algorithms and the accuracy of the separated results are improved observably.

Abstract: We present a novel maximum likelihood sequence detection (MLSD) receiver structure for nonlinear channels. This scheme is derived by treating the NLC as a multiple input/multiple output system. Then, orthogonal signal components are computed using a special form of space-time whitened matched filter (ST-WMF) obtained by a modified Gram-Schmidt orthogonalization of the Volterra kernels of the NLC. The MLSD receiver consists of the ST-WMF followed by a Viterbi detector (VD) with multidimensional branch metrics. The space orthogonalization and noise whitening achieved by the ST-WMF provide an efficient way to reduce the receiver complexity in the presence of highly dispersive NLC. Complexity reduction is crucial in practical applications such as intensity modulation/direct detection (IM/DD) optical channels. As an example, the number of states of the VD in ST-WMF-MLSD required on a 10 Gb/s, 700 km, IM/DD fiber-optic link is reduced eight times compared with an oversampled MLSD.

Abstract: We explore the link between data representation and soft errors in dot products. We present an analytic model for the absolute error introduced should a soft error corrupt a bit in an IEEE-754 floating-point number. We show how this finding relates to the fundamental linear algebra concepts of normalization and matrix equilibration. We present a case study illustrating that the probability of experiencing a large error in a dot product is minimized when both vectors are normalized. Furthermore, when data is normalized we show that the absolute error is less than one or very large, which allows us to detect large errors. We demonstrate how this finding can be used by instrumenting the GMRES iterative solver. We count all possible errors that can be introduced through faults in arithmetic in the computationally intensive orthogonalization phase, and show that when scaling is used the absolute error can be bounded above by one.

Abstract: We propose a mixed-precision orthogonalization scheme that takes the input matrix in a standard 64-bit floating-point precision, but accumulates its intermediate results in the doubled-precision. When the target hardware does not support the desired higher precision, we use software emulation. Compared with the standard orthogonalization scheme, we require about 8.5× more computation but a much smaller increase in communication. Since the computation is becoming less expensive compared to the communication on new and emerging architectures, the relative cost of our mixed-precision scheme is decreasing. Our case studies with CA-GMRES on a GPU demonstrate that using mixed-precision for this small but critical segment of CA-GMRES can improve not only its overall numerical stability but also, in some cases, its performance. We also study an adaptive scheme to dynamically adjust the step size of the matrix powers kernel. Our experiments on multiple GPUs show that a near optimal step size can be chosen based on the performance measurements from the first restart loop of CA-GMRES.

Abstract: It is well known that, in any inner product space, a set of linearly independent (LI) vectors can be transformed to a set of orthogonal vectors, spanning the same space, by the Gram-Schmidt Orthogonalization Method (GSOM). In this paper, we propose a transformation from a set of LI vectors to a set of LI non orthogonal yet energy (square of the norm) preserving (LINOEP) vectors in an inner product space and we refer it as LINOEP method. We also show that there are various solutions to preserve the square of the norm.

Abstract: The Gram–Schmidt process is a technique for constructing an orthogonal basis from a basis spanning the same subspace. The technique is used in a number of multivariate applications including battery reduction analyses and multiple regression applications. In the former, the goal is to reduce the number of variables at the same time explaining as much variance in the original variables as possible. In the latter, the goal is to reduce the n-dimensional space (observations) to a p-dimensional space (independent variables). Keywords: orthogonalization; principal components analysis; battery reduction

Abstract: We investigate the algebra of repeated integrals of semimartingales. We prove that a minimal family of semimartingales generates a quasi-shuffle algebra. In essence, to fulfil the minimality criterion, first, the family must be a minimal generator of the algebra of repeated integrals generated by its elements and by quadratic covariation processes recursively constructed from the elements of the family. Second, recursively constructed quadratic covariation processes may lie in the linear span of previously constructed quadratic covariation processes and of the family, but may not lie in the linear span of repeated integrals of these. We prove that a finite family of independent Levy processes that have finite moments generates a minimal family. Key to the proof are the Teugels martingales and a strong orthogonalization of them. We conclude that a finite family of independent Levy processes forms a quasi-shuffle algebra. We discuss important potential applications to constructing efficient numerical method...

Abstract: The problem of equilibrium of hollow cylinders whose cross sections have the shape of connected semicorrugations is studied in the three-dimensional formulation. The cross section is specified by a continuous curve (shortened epicycloid). The problem is solved by using a numerical-analytical approach based on the application of the methods of separation of variables and approximation of functions by discrete Fourier series and the numerical method of discrete orthogonalization. The results of solution of the problem are presented in the form of the plots of the fields of stresses and displacements.

Abstract: This paper presents knowledge-aided space-time adaptive processing (KA-STAP) algorithms that exploit the low-rank dominant clutter and the array geometry properties (LRGP) for airborne radar applications The core idea is to exploit the clutter subspace that is only determined by the space-time steering vectors, by employing the Gram-Schmidt orthogonalization approach to compute the clutter subspace Simulation results illustrate the effectiveness of our proposed algorithms

Abstract: In [SIAM J. Matrix Anal. Appl., 31 (2009), pp. 565--583] it was shown how a special $(n\!+\!k)\times (n\!+\!k)$ unitary matrix can be defined from any sequence of $k$ vectors in $\mathbb{C}^n$ having unit Euclidean norms. This unitary matrix can be called an augmented orthogonal matrix when applied in the analysis of any algorithm that seeks to compute $k$ orthonormal $n$-vectors, but where the computed, then theoretically normalized, vectors $v_j$ in $V_k=[v_1,\ldots,v_k]$ have a significant loss of orthogonality. These unitary matrices can occur in other situations, being in fact products of $k$ particular Householder matrices (unitary elementary Hermitians), and they have many interesting theoretical properties. Several new results concerning them have been collected here so that they can be easily referenced, our main purpose being to facilitate the rounding error analyses of iterative orthogonalization algorithms which lose significant orthogonality, such as the Lanczos process and its many related p...

Abstract: A Toeplitz matrix has constant diagonals; a multilevel Toeplitz matrix is defined recursively with respect to the levels by replacing the matrix elements with Toeplitz blocks. Multilevel Toeplitz linear systems appear in a wide range of applications in science and engineering. This paper discusses an MPI implementation for solving such a linear system by using the conjugate gradient algorithm. The implementation techniques can be generalized to other iterative Krylov methods besides conjugate gradient. These techniques include the use of an arbitrary dimensional process grid for handling the multilevel Toeplitz structure, a communication-hiding approach for performing matrix–vector multiplications, the incorporation of multilevel circulant preconditioning for accelerating convergence, an efficient orthogonalization manager for detecting linear dependence in block iterations, and an algorithmic rearrangement to eliminate all-reduce synchronizations. The combined use of these techniques leads to a scalable solver for large multilevel Toeplitz systems, possibly with several right-hand sides. We show experimental results on matrices of size up to the order of one billion with nearly perfect scaling by using up to 1024 MPI processes. We also demonstrate an application of the solver in parameter estimation for analyzing large-scale climate data.

Abstract: The invention relates to a gear fault diagnosis method based on the orthogonal match between multiple parallel dictionaries. According to the method, gear vibration signals are expressed in the mode of linear superposition of simple and sparse atoms of the multiple parallel dictionaries. As for the multiple parallel dictionaries, Fourier dictionaries and impact time frequency dictionaries are selected to form the multiple dictionaries according to the characteristics of the gear vibration signals, matched atoms are selected in parallel in all sub-dictionaries with a genetic algorithm, coefficients of all orders are compared to obtain a most matched atom, Gram-Schmidt orthogonalization is performed on the atom, and then a new atom library is formed. Analysis signals are projected to the atom library, and the projections are subtracted from the signals to form residual signals to be decomposed the next time. The decomposition process is completed after the iteration end conditions are met, the matched atoms and the matching coefficient are extracted, the matched atoms based on the impact time frequency dictionaries are reconstructed, corresponding impact components can be obtained, and then fault information of the gear vibration signals is demodulated and extracted for fault diagnosis.

Abstract: The performance of an algorithm on any architecture is dependent on the processing unit’s speed for performing floating point operations (flops) and the speed of accessing memory and disk. As the cost of communication is much higher than arithmetic operations, and since this gap is expected to continue to increase exponentially, communication is often the bottleneck in numerical algorithms. In a quest to address the communication problem, recent research has focused on communication avoiding Krylov subspace methods based on the so called s-step methods. However there are very few communication avoiding preconditioners, and this represents a serious limitation of these methods. In this thesis, we present a communication avoiding ILU0 preconditioner for solving large systems of linear equations (Ax=b) by using iterative Krylov subspace methods. Our preconditioner allows to perform s iterations of the iterative method with no communication, by applying a heuristic alternating min-max layers reordering to the input matrix A, and through ghosting some of the input data and performing redundant computation. We also introduce a new approach for reducing communication in the Krylov subspace methods, that consists of enlarging the Krylov subspace by a maximum of t vectors per iteration, based on the domain decomposition of the graph of A. The enlarged Krylov projection subspace methods lead to faster convergence in terms of iterations and to parallelizable algorithms with less communication, with respect to Krylov methods. We discuss two new versions of Conjugate Gradient, multiple search direction with orthogonalization CG (MSDO-CG) and long recurrence enlarged CG (LRE-CG).

Abstract: The generalized minimum residual (GMRES) method is a popular method for solving a large-scale sparse nonsymmetric linear system of equations On modern computers, especially on a large-scale system, the communication is becoming increasingly expensive To address this hardware trend, a communication-avoiding variant of GMRES (CA-GMRES) has become attractive, frequently showing its superior performance over GMRES on various hardware architectures In practice, to mitigate the increasing costs of explicitly orthogonalizing the projection basis vectors, the iterations of both GMRES and CA-GMRES are restarted, which often slows down the solution convergence To avoid this slowdown and improve the performance of restarted CA-GMRES, in this paper, we study the effectiveness of deflation strategies Our studies are based on a thick restarted variant of CA-GMRES, which can implicitly deflate a few Ritz vectors, that approximately span an eigenspace of the coefficient matrix, through the standard orthogonalization process This strategy is mathematically equivalent to the standard thick-restarted GMRES, and it requires only a small computational overhead and does not increase the communication or storage costs of CA-GMRES Hence, by avoiding the communication, this deflated version of CA-GMRES obtains the same performance benefits over the deflated version of GMRES as the standard CA-GMRES does over GMRES Our experimental results on a hybrid CPU/GPU cluster demonstrate that thick-restart can significantly improve the convergence and reduce the solution time of CA-GMRES We also show that this deflation strategy can be combined with a local domain decomposition based preconditioner to further enhance the robustness of CA-GMRES, making it more attractive in practice

Abstract: The aim of this expository/pedagogical paper is to describe a Gram-Schmidt biorthogonalization method in such a way that it can be used as an introduction to the subject for undergraduate presentation. The task of biorthogonalization naturally arises when the scalar product of vectors formed are linear combinations of two sets of linearly independent vectors, as the case may be. If one wants the scalar product to have the usual form, the two sets of basis vectors should be biorthogonal. If they are not, the question of biorthogonalization arises. New is the detailed description of the biorthogonalzation method for teaching purposes as well as the comparison of this method with Schmidt's orthogonalization method in the case when the sets of linearly independent vectors are identical.

Abstract: The FEAST method for solving large sparse eigenproblems is equivalent to subspace iteration with an approximate spectral projector and implicit orthogonalization. This relation allows to characterize the convergence of this method in terms of the error of a certain rational approximant to an indicator function. We propose improved rational approximants leading to FEAST variants with faster convergence, in particular, when using rational approximants based on the work of Zolotarev. Numerical experiments demonstrate the possible computational savings especially for pencils whose eigenvalues are not well separated and when the dimension of the search space is only slightly larger than the number of wanted eigenvalues. The new approach improves both convergence robustness and load balancing when FEAST runs on multiple search intervals in parallel.

Abstract: This work presents a novel bases selection approach for acoustic model interpolation based fast on-line adaptation. The proposed approach employs a correlation based similarity measure in the supervector domain (derived by concatenating the Gaussian mean parameters of the adapted models) for the selection of bases. This approach is found to greatly reduce the computational complexity in comparison to the Viterbi-alignment based bases search. Moreover, the proposed approach employs joint representation along with orthogonalization for the dynamic selection of bases. Consequently, the selected bases result in a much balanced coverage of phonetic contexts in the synthesized adapted model. The proposed technique is found to result in improved performance for all three modes of adaptation viz. the utterance-specific, the incremental and the batch modes. For utterance-specific mode, it achieves a relative improvement of 10.2% over baseline with only 3 to 5 seconds of adaptation data.

Abstract: Silicon-on-insulator (SOI) waveguides are the focus of present-day photonics due to their smaller device footprint and compatibility with CMOS technology. Coupled waveguide arrays have been widely analyzed by the simple yet intuitive coupled-mode theory. However, for asymmetric strongly coupled waveguide arrays, the supermodes obtained by coupled-mode theory are not orthogonal solutions, thereby introducing error in the power distribution in the array. We present an elegant, completely generalized three-step analytical methodology for obtaining accurate orthogonal supermodes. Once the propagation characteristics of the individual waveguides are obtained, an orthogonal basis set, which is formed by a linear combination of the modal fields of the individual waveguides, is generated using the Gram–Schmidt orthogonalization procedure. The Ritz–Galerkin variational method, with the trial field as an expansion in terms of the newly formed orthogonal basis set, is then used to obtain the modes of the coupled waveguide array. The procedure is illustrated by use on strongly coupled longitudinally homogenous 2D asymmetric SOI waveguide arrays. The TE modal solutions obtained for the waveguide arrays are exactly orthogonal to one another. Since the new orthogonal basis set is essentially a linear combination of the modal solutions of the individual waveguides, the quantities involved in the analysis are analytically identical to those defined as coupling coefficients and overlap integrals in the conventional coupled-mode theory. The excitation and power distribution along the propagation distance obtained from the proposed method is simple and accurate. The theory presents itself as a strong alternative to numerically intensive and time-consuming techniques that are frequently employed for the analysis of such structures.

Abstract: We propose a method of the asynchronous sum-of-products SOP logic simplification that comprises of minimization and orthogonalization. The method is based on a transformation of the conventional single-rail SOP synchronous logic into the dual-rail asynchronous one operating under so-called modified weak constraints. We formulate and prove the product terms constraint that ensures a correct logic behavior. We have processed the MCNC benchmarks and generated the asynchronous SOP logic. The complexity of the logic obtained is compared with the state-of-the-art representation. Using our approach, we achieve a significant improvement. Copyright © 2012 John Wiley & Sons, Ltd.