Abstract: Iterative algorithms for nonexpansive mappings and maximal monotone operators are investigated. Strong convergence theorems are proved for nonexpansive mappings, including an improvement of a result of Lions. A modification of Rockafellar’s proximal point algorithm is obtained and proved to be always strongly convergent. The ideas of these algorithms are applied to solve a quadratic minimization problem.

Abstract: Let C and Q be nonempty closed convex sets in R N and R M , respectively, and A an M by N real matrix. The split feasibility problem (SFP) is to find x ∈ C with Ax ∈ Q ,i f suchx exist. An iterative method for solving the SFP, called the CQ algorithm, has the following iterative step: x k+1 = PC (x k + γ A T (PQ − I )Ax k ), where γ ∈ (0, 2/L) with L the largest eigenvalue of the matrix A T A and PC and PQ denote the orthogonal projections onto C and Q, respectively; that is, PC x minimizesc − x� ,o ver allc ∈ C.T heCQ algorithm converges to a solution of the SFP, or, more generally, to a minimizer ofPQ Ac − Acover c in C, whenever such exist. The CQ algorithm involves only the orthogonal projections onto C and Q, which we shall assume are easily calculated, and involves no matrix inverses. If A is normalized so that each row has length one, then L does not exceed the maximum number of nonzero entries in any column of A, which provides a helpful estimate of L for sparse matrices. Particular cases of the CQ algorithm are the Landweber and projected Landweber methods for obtaining exact or approximate solutions of the linear equations Ax = b ;t healgebraic reconstruction technique of Gordon, Bender and Herman is a particular case of a block-iterative version of the CQ algorithm. One application of the CQ algorithm that is the subject of ongoing work is dynamic emission tomographic image reconstruction, in which the vector x is the concatenation of several images corresponding to successive discrete times. The matrix A and the set Q can then be selected to impose constraints on the behaviour over time of the intensities at fixed voxels, as well as to require consistency (or near consistency) with measured data.

Abstract: We study efficient iterative methods for the large sparse non-Hermitian positive definite system of linear equations based on the Hermitian and skew-Hermitian splitting of the coefficient matrix. These methods include a Hermitian/skew-Hermitian splitting (HSS) iteration and its inexact variant, the inexact Hermitian/skew-Hermitian splitting (IHSS) iteration, which employs some Krylov subspace methods as its inner iteration processes at each step of the outer HSS iteration. Theoretical analyses show that the HSS method converges unconditionally to the unique solution of the system of linear equations. Moreover, we derive an upper bound of the contraction factor of the HSS iteration which is dependent solely on the spectrum of the Hermitian part and is independent of the eigenvectors of the matrices involved. Numerical examples are presented to illustrate the effectiveness of both HSS and IHSS iterations. In addition, a model problem of a three-dimensional convection-diffusion equation is used to illustrate the advantages of our methods.

Abstract: Additive noise removal from a given signal is an important problem in signal processing. Among the most appealing aspects of this field are the ability to refer it to a well-established theory, and the fact that the proposed algorithms in this field are efficient and practical. Adaptive methods based on anisotropic diffusion (AD), weighted least squares (WLS), and robust estimation (RE) were proposed as iterative locally adaptive machines for noise removal. Tomasi and Manduchi (see Proc. 6th Int. Conf. Computer Vision, New Delhi, India, p.839-46, 1998) proposed an alternative noniterative bilateral filter for removing noise from images. This filter was shown to give similar and possibly better results to the ones obtained by iterative approaches. However, the bilateral filter was proposed as an intuitive tool without theoretical connection to the classical approaches. We propose such a bridge, and show that the bilateral filter also emerges from the Bayesian approach, as a single iteration of some well-known iterative algorithm. Based on this observation, we also show how the bilateral filter can be improved and extended to treat more general reconstruction problems.

Abstract: Image processing methods for particle image velocimetry (PIV) are reviewed. The discussion focuses on iterative methods aimed at enhancing the precision and spatial resolution of numerical interrogation schemes. Emphasis is placed on the efforts made to overcome the limitations of the correlation interrogation method with respect to typical problems such as in-plane loss of pairs, velocity gradient compensation and correlation peak locking. The discussion shows how the correlation signal benefits from simple operations such as the window-offset technique, or the continuous window deformation, which compensates for the in-plane velocity gradient. The image interrogation process is presented within the discussion of the image matching problem and several algorithms and implementations currently in use are classified depending on the choice made about the particle image pattern matching scheme. Several methods that differ in their implementations are found to be substantially similar. Iterative image deformation methods that account for the continuous particle image pattern transformation are analysed and the effect of crucial choices such as image interpolation method, displacement prediction correction and correlation peak fit scheme are discussed. The quantitative performance assessment made through synthetic PIV images yields order-of-magnitude improvement on the precision of the particle image displacement at a sub-pixel level when the image deformation is applied. Moreover, the issue of spatial resolution is addressed and the limiting factors of the specific interrogation methods are discussed. Finally, an attempt for a flow-adaptive spatial resolution method is proposed. The method takes into account the local velocity derivatives in order to perform a local increase of the interrogation spatial density and a refinement of the local interrogation window size. The resulting spatial resolution is selectively enhanced. The method's performance is analysed and compared with some precursor techniques, namely the conventional cross-correlation analysis with and without the effect of a window discrete offset and deformation. The suitability of the method for the measurement in turbulent flows is illustrated with the application to a turbulent backward facing step flow.

Abstract: Presents a framework for three-dimensional model-based tracking. Graphical rendering technology is combined with constrained active contour tracking to create a robust wire-frame tracking system. It operates in real time at video frame rate (25 Hz) on standard hardware. It is based on an internal CAD model of the object to be tracked which is rendered using a binary space partition tree to perform hidden line removal. A Lie group formalism is used to cast the motion computation problem into simple geometric terms so that tracking becomes a simple optimization problem solved by means of iterative reweighted least squares. A visual servoing system constructed using this framework is presented together with results showing the accuracy of the tracker. The paper then describes how this tracking system has been extended to provide a general framework for tracking in complex configurations. The adjoint representation of the group is used to transform measurements into common coordinate frames. The constraints are then imposed by means of Lagrange multipliers. Results from a number of experiments performed using this framework are presented and discussed.

Abstract: Investigates the use of Euclidean invariant features in a generalization of iterative closest point (ICP) registration of range images. Pointwise correspondences are chosen as the closest point with respect to a weighted linear combination of positional and feature distances. It is shown that, under ideal noise-free conditions, correspondences formed using this distance function are correct more often than correspondences formed using the positional distance alone. In addition, monotonic convergence to at least a local minimum is shown to hold for this method. When noise is present, a method that automatically sets the optimal relative contribution of features and positions is described. This method trades off the error in feature values due to noise against the error in positions due to misalignment. Experimental results suggest that using invariant features decreases the probability of being trapped in a local minimum and may be an effective solution for difficult range image registration problems where the scene is very small compared to the model.

Abstract: We consider the problem of maximizing sum rate on a multiple-antenna downlink in which the base station and receivers have multiple-antennas. The optimum scheme for this system was recently found to be "dirty paper coding". Obtaining the optimal transmission policies of the users when employing this dirty paper coding scheme is a computationally complex nonconvex problem. We use a "duality" to transform this problem into a convex multiple access problem, and then obtain a simple and fast iterative algorithm that gives us the optimum transmission policies.

Abstract: The classical Schwarz method is a domain decomposition method to solve elliptic partial differential equations in parallel. Convergence is achieved through overlap of the subdomains. We study in this paper a variant of the Schwarz method which converges without overlap for the Helmholtz equation. We show that the key ingredients for such an algorithm are the transmission conditions. We derive optimal transmission conditions which lead to convergence of the algorithm in a finite number of steps. These conditions are, however, nonlocal in nature, and we introduce local approximations which we optimize for performance of the Schwarz method. This leads to an algorithm in the class of optimized Schwarz methods. We present an asymptotic analysis of the optimized Schwarz method for two types of transmission conditions, Robin conditions and transmission conditions with second order tangential derivatives. Numerical results illustrate the effectiveness of the optimized Schwarz method on a model problem and on a problem from industry.

Abstract: Images of the inside of the human body can be obtained noninvasively using tomographic acquisition and processing techniques. In particular, these techniques are commonly used to obtain images of a γ-emitter distribution after its administration in the human body. The reconstructed images are obtained given a set of their projections, acquired using rotating gamma cameras. A general overview of analytic and iterative methods of reconstruction in SPECT is presented with a special focus on filter backprojection (FBP), conjugate gradient, maximum likelihood expectation maximization, and maximum a posteriori expectation maximization algorithms. The FBP algorithm is faster than iterative algorithms, with the latter providing a framework for accurately modeling the emission and detection processes.

Abstract: A modification is given of the GMRES iterative method for nonsymmetric systems of linear equations. The new method deflates eigenvalues using Wu and Simon's thick restarting approach [SIAM J. Matrix Anal. Appl., 22 (2000), pp. 602--616]. It has the efficiency of implicit restarting but is simpler and does not have the same numerical concerns. The deflation of small eigenvalues can greatly improve the convergence of restarted GMRES. Also, it is demonstrated that using harmonic Ritz vectors is important because then the whole subspace is a Krylov subspace that contains certain important smaller subspaces.

Abstract: A direct inversion iterative subspace version of the relaxed constrained algorithm is found to be a very powerful convergence acceleration technique for the solution of the self-consistent field equations found in the Hartree–Fock method and Kohn–Sham-based density functional theory (KS-DFT). The present algorithm, abbreviated EDIIS, is benchmarked against the direct inversion iterative subspace method based on the commutator of the density and Fock matrices developed by Pulay (DIIS). Our findings indicate that while EDIIS is able to rapidly bring the density matrix from any initial guess to a solution region, the DIIS method is faster when the density matrix is close to convergence. Consequently, we propose a combination of EDIIS and DIIS methods, which is both very robust and highly efficient. We also show how EDIIS can detect the presence and determine the value of fractional occupations in KS-DFT.

Abstract: The selection of multiple regularization parameters is considered in a generalized L-curve framework. Multiple-dimensional extensions of the L-curve for selecting multiple regularization parameters are introduced, and a minimum distance function (MDF) is developed for approximating the regularization parameters corresponding to the generalized corner of the L-hypersurface. For the single-parameter (i.e. L-curve) case, it is shown through a model that the regularization parameters minimizing the MDF essentially maximize the curvature of the L-curve. Furthermore, for both the single-and multiple-parameter cases the MDF approach leads to a simple fixed-point iterative algorithm for computing regularization parameters. Examples indicate that the algorithm converges rapidly thereby making the problem of computing parameters according to the generalized corner of the L-hypersurface computationally tractable.

Abstract: In this paper, certain iterative substructuring methods with Lagrange multipliers are considered for elliptic problems in three dimensions. The algorithms belong to the family of dual-primal finite element tearing and interconnecting (FETI) methods which recently have been introduced and analyzed successfully for elliptic problems in the plane. The family of algorithms for three dimensions is extended and a full analysis is provided for the new algorithms. Particular attention is paid to finding algorithms with a small primal subspace since that subspace represents the only global part of the dual-primal preconditioner. It is shown that the condition numbers of several of the dual-primal FETI methods can be bounded polylogarithmically as a function of the dimension of the individual subregion problems and that the bounds are otherwise independent of the number of subdomains, the mesh size, and jumps in the coefficients. These results closely parallel those of other successful iterative substructuring methods of primal as well as dual type.

Abstract: Traditional methods for characterizing an optimized molecular structure as a minimum or as a saddle point on the nuclear potential energy surface require the full Hessian. However, if f denotes the number of nuclear degrees of freedom, a full Hessian calculation is more expensive than a single point geometry optimization step by the order of magnitude of f. Here we present a method which allows to determine the lowest vibrational frequencies of a molecule at significantly lower cost. Our approach takes advantage of the fact that only a few perturbed first-order wave functions need to be computed in an iterative diagonalization scheme instead of f ones in a full Hessian calculation. We outline an implementation for Hartree–Fock and density functional methods. Applications indicate a scaling similar to that of a single point energy or gradient calculation, but with a larger prefactor. Depending on the number of soft vibrational modes, the iterative method becomes effective for systems with more than 30–50 atoms.

Abstract: The article considers linear systems subject to actuator saturation. An antiwindup technique is used to enlarge the domain of attraction of the closed-loop system under an a priori designed linear dynamic feedback law. The design of the antiwindup compensation gain is formulated and solved as an iterative optimization problem with LMI constraints. Numerical examples are used to demonstrate the effectiveness of the proposed design technique.

Abstract: The bidomain equations are the most complete description of cardiac electrical activity. Their numerical solution is, however, computationally demanding, especially in three dimensions, because of the fine temporal and spatial sampling required. This paper methodically examines computational performance when solving the bidomain equations. Several techniques to speed up this computation are examined in this paper. The first step was to recast the equations into a parabolic part and an elliptic part. The parabolic part was solved by either the finite-element method (FEM) or the interconnected cable model (ICCM). The elliptic equation was solved by FEM on a coarser grid than the parabolic problem and at a reduced frequency. The performance of iterative and direct linear equation system solvers was analyzed as well as the scalability and parallelizability of each method. Results indicate that the ICCM was twice as fast as the FEM for solving the parabolic problem, but when the total problem was considered, this resulted in only a 20% decrease in computation time. The elliptic problem could be solved on a coarser grid at one-quarter of the frequency at which the parabolic problem was solved and still maintain reasonable accuracy. Direct methods were faster than iterative methods by at least 50% when a good estimate of the extracellular potential was required. Parallelization over four processors was efficient only when the model comprised at least 500 000 nodes. Thus, it was possible to speed up solution of the bidomain equations by an order of magnitude with a slight decrease in accuracy.

Abstract: We examine the convergence characteristics of iterative methods based on a new preconditioning operator for solving the linear systems arising from discretization and linearization of the steady-state Navier-Stokes equations. With a combination of analytic and empirical results, we study the effects of fundamental parameters on convergence. We demonstrate that the preconditioned problem has an eigenvalue distribution consisting of a tightly clustered set together with a small number of outliers. The structure of these distributions is independent of the discretization mesh size, but the cardinality of the set of outliers increases slowly as the viscosity becomes smaller. These characteristics are directly correlated with the convergence properties of iterative solvers.

Abstract: generates a sequence {xn}n=0 that converges to ζ. In fact, Newton’s original ideas on the subject, around 1669, were considerably more complicated. A systematic study and a simplified version of the method are due to Raphson in 1690, so this iteration scheme is also known as the Newton-Raphson method. (Also as the tangent method, from its geometric interpretation.) In 1879, Cayley tried to use the method to find complex roots of complex functions f : C → C. If we take z0 ∈ C and we iterate

Abstract: We present a fast iterative algorithm for identifying the support vectors of a given set of points. Our algorithm works by maintaining a candidate support vector set. It uses a greedy approach to pick points for inclusion in the candidate set. When the addition of a point to the candidate set is blocked because of other points already present in the set, we use a backtracking approach to prune away such points. To speed up convergence we initialize our algorithm with the nearest pair of points from opposite classes. We then use an optimization based approach to increase or prune the candidate support vector set. The algorithm makes repeated passes over the data to satisfy the KKT constraints. The memory requirements of our algorithm scale as O(|SI|/sup 2/) in the average case, where |S| is the size of the support vector set. We show that the algorithm is extremely competitive as compared to other conventional iterative algorithms like SMO and the NPA. We present results on a variety of real life datasets to validate our claims.

Abstract: The major concerns in state-of-the-art model reduction algorithms are: achieving accurate models of sufficiently small size, numerically stable and efficient generation of the models, and preservation of system properties such as passivity. Algorithms such as PRIMA generate guaranteed-passive models, for systems with special internal structure, using numerically stable and efficient Krylov-subspace iterations. Truncated balanced realization (TBR) algorithms, as used to date in the design automation community, can achieve smaller models with better error control, but do not necessarily preserve passivity. In this paper we show how to construct TBR-like methods that guarantee passive reduced models and in addition are applicable to state-space systems with arbitrary internal structure.

Abstract: In this paper two classes of iterative methods for saddle point problems are considered: inexact Uzawa algorithms and a class of methods with symmetric preconditioners. In both cases the iteration matrix can be transformed to a symmetric matrix by block diagonal matrices, a simple but essential observation which allows one to estimate the convergence rate of both classes by studying associated eigenvalue problems. The obtained estimates apply for a wider range of situations and are partially sharper than the known estimates in literature. A few numerical tests are given which confirm the sharpness of the estimates.

Abstract: By deconvolution we mean the solution of a linear first-kind integral equation with a convolution-type kernel, i.e., a kernel that depends only on the difference between the two independent variables. Deconvolution problems are special cases of linear first-kind Fredholm integral equations, whose treatment requires the use of regularization methods. The corresponding computational problem takes the form of structured matrix problem with a Toeplitz or block Toeplitz coefficient matrix. The aim of this paper is to present a tutorial survey of numerical algorithms for the practical treatment of these discretized deconvolution problems, with emphasis on methods that take the special structure of the matrix into account. Wherever possible, analogies to classical DFT-based deconvolution problems are drawn. Among other things, we present direct methods for regularization with Toeplitz matrices, and we show how Toeplitz matrix–vector products are computed by means of FFT, being useful in iterative methods. We also introduce the Kronecker product and show how it is used in the discretization and solution of 2-D deconvolution problems whose variables separate.

Abstract: [1] The integral equation method has been proven to be an efficient tool to model three-dimensional electromagnetic problems. Owing to the full linear system to be solved, the method has been considered effective only in the case of models consisting of a strongly limited number of cells. However, recent advances in matrix storage and multiplication issues facilitate the modeling of horizontally large structures. Iterative methods are the most feasible techniques for obtaining accurate solutions for such problems. In this paper we demonstrate that the convergence of iterative methods can be improved significantly, if the original integral equation is replaced by an equation based on the modified Green's operator with the norm less or equal to one. That is why we call this technique the Contraction Integral Equation (CIE) method. We demonstrate that application of the modified Green's operator can be treated as a preconditioning of the original problem. We have performed a comparative study of the convergence of different iterative solvers applied to the original and contraction integral equations. The results show that the most effective solvers are the BIGGSTAB, QMRCGSTAB, and CGMRES algorithms, equipped with preconditioning based on the CIE method.

Abstract: A new O(N ) method for the iterative treatment of connected triple substitutions in the framework of local coupled cluster theory is introduced here, which is the local equivalent of the canonical CCSDT-1b method. The effect of the triple substitutions is treated in a self-consistent manner in each coupled cluster iteration. As for the local (T) method presented earlier in this series the computational cost of the method scales asymptotically linear with molecular size. The additional computational burden due to the involvement of triples in each coupled cluster iteration hence is not nearly as dramatic as for the parental canonical method, where it implies an increase in the computational complexity of the coupled cluster iteration from O(N6) to O(N7). The method has certain advantages in comparison to the perturbative a posteriori treatment of connected triples (T) for cases where static correlation effects start to play a role. It is demonstrated that molecules with about 100 atoms and 1000 basis funct...