Abstract: This paper is concerned with a new discrete-time policy iteration adaptive dynamic programming (ADP) method for solving the infinite horizon optimal control problem of nonlinear systems. The idea is to use an iterative ADP technique to obtain the iterative control law, which optimizes the iterative performance index function. The main contribution of this paper is to analyze the convergence and stability properties of policy iteration method for discrete-time nonlinear systems for the first time. It shows that the iterative performance index function is nonincreasingly convergent to the optimal solution of the Hamilton-Jacobi-Bellman equation. It is also proven that any of the iterative control laws can stabilize the nonlinear systems. Neural networks are used to approximate the performance index function and compute the optimal control law, respectively, for facilitating the implementation of the iterative ADP algorithm, where the convergence of the weight matrices is analyzed. Finally, the numerical results and analysis are presented to illustrate the performance of the developed method.

Abstract: In this paper, two online schemes for an integrated design of fault-tolerant control (FTC) systems with application to Tennessee Eastman (TE) benchmark are proposed. Based on the data-driven design of the proposed fault-tolerant architecture whose core is an observer/residual generator based realization of the Youla parameterization of all stabilization controllers, FTC is achieved by an adaptive residual generator for the online identification of the fault diagnosis relevant vectors, and an iterative optimization method for system performance enhancement. The performance and effectiveness of the proposed schemes are demonstrated through the TE benchmark model.

Abstract: This volume presents a unified approach to constructing iterative methods for solving irregular operator equations and provides rigorous theoretical analysis for several classes of these methods. The analysis of methods includes convergence theorems as well as necessary and sufficient conditions for their convergence at a given rate. The principal groups of methods studied in the book are iterative processes based on the technique of universal linear approximations, stable gradient-type processes, and methods of stable continuous approximations. Compared to existing monographs and textbooks on ill-posed problems, the main distinguishing feature of the presented approach is that it doesnt require any structural conditions on equations under consideration, except for standard smoothness conditions. This allows to obtain in a uniform style stable iterative methods applicable to wide classes of nonlinear inverse problems. Practical efficiency of suggested algorithms is illustrated in application to inverse problems of potential theory and acoustic scattering. The volume can be read by anyone with a basic knowledge of functional analysis. The book will be of interest to applied mathematicians and specialists in mathematical modeling and inverse problems.

Abstract: Robust sparse representation has shown significant potential in solving challenging problems in computer vision such as biometrics and visual surveillance. Although several robust sparse models have been proposed and promising results have been obtained, they are either for error correction or for error detection, and learning a general framework that systematically unifies these two aspects and explores their relation is still an open problem. In this paper, we develop a half-quadratic (HQ) framework to solve the robust sparse representation problem. By defining different kinds of half-quadratic functions, the proposed HQ framework is applicable to performing both error correction and error detection. More specifically, by using the additive form of HQ, we propose an l1-regularized error correction method by iteratively recovering corrupted data from errors incurred by noises and outliers; by using the multiplicative form of HQ, we propose an l1-regularized error detection method by learning from uncorrupted data iteratively. We also show that the l1-regularization solved by soft-thresholding function has a dual relationship to Huber M-estimator, which theoretically guarantees the performance of robust sparse representation in terms of M-estimation. Experiments on robust face recognition under severe occlusion and corruption validate our framework and findings.

Abstract: The authors introduce an iterative algorithm, called matching demodulation transform (MDT), to generate a time-frequency (TF) representation with satisfactory energy concentration. As opposed to conventional TF analysis methods, this algorithm does not have to devise ad-hoc parametric TF dictionary. Assuming the FM law of a signal can be well characterized by a determined mathematical model with reasonable accuracy, the MDT algorithm can adopt a partial demodulation and stepwise refinement strategy for investigating TF properties of the signal. The practical implementation of the MDT involves an iterative procedure that gradually matches the true instantaneous frequency (IF) of the signal. Theoretical analysis of the MDT's performance is provided, including quantitative analysis of the IF estimation error and the convergence condition. Moreover, the MDT-based synchrosqueezing algorithm is described to further enhance the concentration and reduce the diffusion of the curved IF profile in the TF representation of original synchrosqueezing transform. The validity and practical utility of the proposed method are demonstrated by simulated as well as real signal.

Abstract: Node self-localization is a key research topic for wireless sensor networks (WSNs). There are two main algorithms, the triangulation method and the maximum likelihood (ML) estimator, for angle of arrival (AOA) based self-localization. The ML estimator requires a good initialization close to the true location to avoid divergence, while the triangulation method cannot obtain the closed-form solution with high efficiency. In this paper, we develop a set of efficient closed-form AOA based self-localization algorithms using auxiliary variables based methods. First, we formulate the self-localization problem as a linear least squares problem using auxiliary variables. Based on its closed-form solution, a new auxiliary variables based pseudo-linear estimator (AVPLE) is developed. By analyzing its estimation error, we present a bias compensated AVPLE (BCAVPLE) to reduce the estimation error. Then we develop a novel BCAVPLE based weighted instrumental variable (BCAVPLE-WIV) estimator to achieve asymptotically unbiased estimation of locations and orientations of unknown nodes based on prior knowledge of the AOA noise variance. In the case that the AOA noise variance is unknown, a new AVPLE based WIV (AVPLE-WIV) estimator is developed to localize the unknown nodes. Also, we develop an autonomous coordinate rotation (ACR) method to overcome the tangent instability of the proposed algorithms when the orientation of the unknown node is near π/2. We also derive the Cramer-Rao lower bound (CRLB) of the ML estimator. Extensive simulations demonstrate that the new algorithms achieve much higher localization accuracy than the triangulation method and avoid local minima and divergence in iterative ML estimators.

Abstract: The book is designed for researchers, students and practitioners interested in using fast and efficient iterative methods to approximate solutions of nonlinear equations. The following four major problems are addressed: problems 1 shows that the iterates are well defined; 2nd problem concerns the convergence of the sequences generated by a process and the question of whether the limit points are, in fact solutions of the equation; problem 3 concerns the economy of the entire operations; and last problem concerns with how to best choose a method, algorithm or software program to solve a specific type of problem and its description of when a given algorithm succeeds or fails.The book contains applications in several areas of applied sciences including mathematical programming and mathematical economics. There is also a huge number of exercises complementing the theory. This book contains the latest convergence results for the iterative methods: Iterative methods with the least computational cost; Iterative methods with the weakest convergence conditions; and Open problems on iterative methods.

Abstract: We show an algorithm for solving symmetric diagonally dominant (SDD) linear systems with m non-zero entries to a relative error of e in O(m log1/2 n logc n log(1/e)) time. Our approach follows the recursive preconditioning framework, which aims to reduce graphs to trees using iterative methods. We improve two key components of this framework: random sampling and tree embeddings. Both of these components are used in a variety of other algorithms, and our approach also extends to the dual problem of computing electrical flows. We show that preconditioners constructed by random sampling can perform well without meeting the standard requirements of iterative methods. In the graph setting, this leads to ultra-sparsifiers that have optimal behavior in expectation. The improved running time makes previous low stretch embedding algorithms the running time bottleneck in this framework. In our analysis, we relax the requirement of these embeddings to snowflake spaces. We then obtain a two-pass approach algorithm for constructing optimal embeddings in snowflake spaces that runs in O(m log log n) time. This algorithm is also readily parallelizable.

Abstract: The NP-hard problem of optimizing a quadratic form over the unimodular vector set arises in radar code design scenarios as well as other active sensing and communication applications. To tackle this problem (which we call unimodular quadratic program (UQP)), several computational approaches are devised and studied. Power method-like iterations are introduced for local optimization of UQP. Furthermore, a monotonically error-bound improving technique (MERIT) is proposed to obtain the global optimum or a local optimum of UQP with good sub-optimality guarantees. The provided sub-optimality guarantees are case-dependent and may outperform the π/4 approximation guarantee of semi-definite relaxation. Several numerical examples are presented to illustrate the performance of the proposed method. The examples show that for several cases, including rank-deficient matrices, the proposed methods can solve UQPs efficiently in the sense of sub-optimality guarantee and computational time.

Abstract: Fisher linear discriminant analysis (LDA) is a classical subspace learning technique of extracting discriminative features for pattern recognition problems. The formulation of the Fisher criterion is based on the L2-norm, which makes LDA prone to being affected by the presence of outliers. In this paper, we propose a new method, termed LDA-L1, by maximizing the ratio of the between-class dispersion to the within-class dispersion using the L1-norm rather than the L2-norm. LDA-L1 is robust to outliers, and is solved by an iterative algorithm proposed. The algorithm is easy to be implemented and is theoretically shown to arrive at a locally maximal point. LDA-L1 does not suffer from the problems of small sample size and rank limit as existed in the conventional LDA. Experiment results of image recognition confirm the effectiveness of the proposed method.

Abstract: In this paper, a novel strategy is established to design the robust controller for a class of continuous-time nonlinear systems with uncertainties based on the online policy iteration algorithm. The robust control problem is transformed into the optimal control problem by properly choosing a cost function that reflects the uncertainties, regulation, and control. An online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman (HJB) equation by constructing a critic neural network. The approximate expression of the optimal control policy can be derived directly. The closed-loop system is proved to possess the uniform ultimate boundedness. The equivalence of the neural-network-based HJB solution of the optimal control problem and the solution of the robust control problem is established as well. Two simulation examples are provided to verify the effectiveness of the present robust control scheme.

Abstract: This technical note addresses an iterative learning control (ILC) design problem for discrete-time linear systems where the trial lengths could be randomly varying in the iteration domain. An ILC scheme with an iteration-average operator is introduced for tracking tasks with non-uniform trial lengths, which thus mitigates the requirement on classic ILC that all trial lengths must be identical. In addition, the identical initialization condition can be absolutely removed. The learning convergence condition of ILC in mathematical expectation is derived through rigorous analysis. As a result, the proposed ILC scheme is applicable to more practical systems. In the end, two illustrative examples are presented to demonstrate the performance and the effectiveness of the averaging ILC scheme for both time-invariant and time-varying linear systems.

Abstract: In this work, an extension is proposed to the standard iterative Boltzmann inversion (IBI) method used to derive coarse-grained potentials. It is shown that the inclusion of target data from multiple states yields a less state-dependent potential, and is thus better suited to simulate systems over a range of thermodynamic states than the standard IBI method. The inclusion of target data from multiple states forces the algorithm to sample regions of potential phase space that match the radial distribution function at multiple state points, thus producing a derived potential that is more representative of the underlying interactions. It is shown that the algorithm is able to converge to the true potential for a system where the underlying potential is known. It is also shown that potentials derived via the proposed method better predict the behavior of n-alkane chains than those derived via the standard IBI method. Additionally, through the examination of alkane monolayers, it is shown that the relative weight given to each state in the fitting procedure can impact bulk system properties, allowing the potentials to be further tuned in order to match the properties of reference atomistic and/or experimental systems.

Abstract: Nonconvex optimization problems arise in many areas of computational science and engineering and are (approximately) solved by a variety of algorithms. Existing algorithms usually only have local convergence or subsequence convergence of their iterates. We propose an algorithm for a generic nonconvex optimization formulation, establish the convergence of its whole iterate sequence to a critical point along with a rate of convergence, and numerically demonstrate its efficiency. Specially, we consider the problem of minimizing a nonconvex objective function. Its variables can be treated as one block or be partitioned into multiple disjoint blocks. It is assumed that each non-differentiable component of the objective function or each constraint applies to one block of variables. The differentiable components of the objective function, however, can apply to one or multiple blocks of variables together. Our algorithm updates one block of variables at time by minimizing a certain prox-linear surrogate. The order of update can be either deterministic or randomly shuffled in each round. We obtain the convergence of the whole iterate sequence under fairly loose conditions including, in particular, the Kurdyka-{\L}ojasiewicz (KL) condition, which is satisfied by a broad class of nonconvex/nonsmooth applications. We apply our convergence result to the coordinate descent method for non-convex regularized linear regression and also a modified rank-one residue iteration method for nonnegative matrix factorization. We show that both the methods have global convergence. Numerically, we test our algorithm on nonnegative matrix and tensor factorization problems, with random shuffling enable to avoid local solutions.

Abstract: Since Moulinec & Suquet (1994, 1998) introduced an iterative method based on Fourier transforms to compute the mechanical properties of heterogeneous materials, improved algorithms have been proposed to increase the convergence rate of the scheme. This paper is devoted to the comparison of the accelerated schemes proposed by Eyre & Milton (1999), by Michel et al (2000) and by Monchiet & Bonnet (2012). It shows that the algorithms by Eyre-Milton and by Michel et al are particular cases of Monchiet-Bonnet algorithm, corresponding to particular choices of parameters of the method. An upper bound of the spectral radius of the schemes is determined, which enables to propose sufficient conditions of convergence of the schemes. Conditions are found for minimizing this upper bound. This study shows that the scheme which minimizes this upper bound is the scheme of Eyre & Milton. The paper discusses the choice of the convergence test used in the schemes.

Abstract: Purpose: Dual energy CT (DECT) imaging plays an important role in advanced imaging applications due to its capability of material decomposition. Direct decomposition via matrix inversion suffers from significant degradation of image signal-to-noise ratios, which reduces clinical values of DECT. Existing denoising algorithms achieve suboptimal performance since they suppress image noise either before or after the decomposition and do not fully explore the noise statistical properties of the decomposition process. In this work, the authors propose an iterative image-domain decomposition method for noise suppression in DECT, using the full variance-covariance matrix of the decomposed images. Methods: The proposed algorithm is formulated in the form of least-square estimation with smoothness regularization. Based on the design principles of a best linear unbiased estimator, the authors include the inverse of the estimated variance-covariance matrix of the decomposed images as the penalty weight in the least-square term. The regularization term enforces the image smoothness by calculating the square sum of neighboring pixel value differences. To retain the boundary sharpness of the decomposed images, the authors detect the edges in the CT images before decomposition. These edge pixels have small weights in the calculation of the regularization term. Distinct from the existing denoising algorithms applied on the images before or after decomposition, the method has an iterative process for noise suppression, with decomposition performed in each iteration. The authors implement the proposed algorithm using a standard conjugate gradient algorithm. The method performance is evaluated using an evaluation phantom (Catphan©600) and an anthropomorphic head phantom. The results are compared with those generated using direct matrix inversion with no noise suppression, a denoising method applied on the decomposed images, and an existing algorithm with similar formulation as the proposed method but with an edge-preserving regularization term. Results: On the Catphan phantom, the method maintains the same spatial resolution on the decomposed images as that of the CT images before decomposition (8 pairs/cm) while significantly reducing their noise standard deviation. Compared to that obtained by the direct matrix inversion, the noise standard deviation in the images decomposed by the proposed algorithm is reduced by over 98%. Without considering the noise correlation properties in the formulation, the denoising scheme degrades the spatial resolution to 6 pairs/cm for the same level of noise suppression. Compared to the edge-preserving algorithm, the method achieves better low-contrast detectability. A quantitative study is performed on the contrast-rod slice of Catphan phantom. The proposed method achieves lower electron density measurement error as compared to that by the direct matrix inversion, and significantly reduces the error variation by over 97%. On the head phantom, the method reduces the noise standard deviation of decomposed images by over 97% without blurring the sinus structures. Conclusions: The authors propose an iterative image-domain decomposition method for DECT. The method combines noise suppression and material decomposition into an iterative process and achieves both goals simultaneously. By exploring the full variance-covariance properties of the decomposed images and utilizing the edge predetection, the proposed algorithm shows superior performance on noise suppression with high image spatial resolution and low-contrast detectability.

Abstract: The traditional metric for the efficiency of a numerical algorithm has been the number of arithmetic operations it performs. Technological trends have long been reducing the time to perform an arithmetic operation, so it is no longer the bottleneck in many algorithms; rather, communication, or moving data, is the bottleneck. This motivates us to seek algorithms that move as little data as possible, either between levels of a memory hierarchy or between parallel processors over a network. In this paper we summarize recent progress in three aspects of this problem. First we describe lower bounds on communication. Some of these generalize known lower bounds for dense classical (O(n3)) matrix multiplication to all direct methods of linear algebra, to sequential and parallel algorithms, and to dense and sparse matrices. We also present lower bounds for Strassen-like algorithms, and for iterative methods, in particular Krylov subspace methods applied to sparse matrices. Second, we compare these lower bounds to widely used versions of these algorithms, and note that these widely used algorithms usually communicate asymptotically more than is necessary. Third, we identify or invent new algorithms for most linear algebra problems that do attain these lower bounds, and demonstrate large speed-ups in theory and practice.

Abstract: Algorithms to solve variational regularization of ill-posed inverse problems usually involve operators that depend on a collection of continuous parameters. When these operators enjoy some (local) regularity, these parameters can be selected using the so-called Stein Unbiased Risk Estimate (SURE). While this selection is usually performed by exhaustive search, we address in this work the problem of using the SURE to efficiently optimize for a collection of continuous parameters of the model. When considering non-smooth regularizers, such as the popular l1-norm corresponding to soft-thresholding mapping, the SURE is a discontinuous function of the parameters preventing the use of gradient descent optimization techniques. Instead, we focus on an approximation of the SURE based on finite differences as proposed in (Ramani et al., 2008). Under mild assumptions on the estimation mapping, we show that this approximation is a weakly differentiable function of the parameters and its weak gradient, coined the Stein Unbiased GrAdient estimator of the Risk (SUGAR), provides an asymptotically (with respect to the data dimension) unbiased estimate of the gradient of the risk. Moreover, in the particular case of soft-thresholding, it is proved to be also a consistent estimator. This gradient estimate can then be used as a basis to perform a quasi-Newton optimization. The computation of the SUGAR relies on the closed-form (weak) differentiation of the non-smooth function. We provide its expression for a large class of iterative methods including proximal splitting ones and apply our strategy to regularizations involving non-smooth convex structured penalties. Illustrations on various image restoration and matrix completion problems are given.

Abstract: We study systems coupled linearly to a bath of oscillators. In an iterative process, the bath is transformed into a chain of oscillators with nearest neighbour interactions. A systematic procedure is provided to obtain the spectral density of the residual bath in each step, and it is shown that under general conditions these data converge. That is, the asymptotic part of the chain is universal, translation invariant with semicircular spectral density. The methods are based on orthogonal polynomials, in which we also solve the outstanding so-called “sequence of secondary measures problem” and give them a physical interpretation.

Abstract: By constructing an objective function and using the gradient search, a gradient-based iteration is established for solving the coupled matrix equations A i XB i = F i , i = 1, 2, …, p The authors prove that the gradient solution is convergent for any initial values By analysing the spectral radius of the iterative matrix, the authors obtain an optimal convergence factor An example is provided to illustrate the effectiveness of the proposed algorithm and to testify the conclusions established in this study

Abstract: In this paper, we introduce and study an iterative method to approximate a common solution of split variational inclusion problem and fixed point problem for a nonexpansive mapping in real Hilbert spaces. Further, we prove that the sequences generated by the proposed iterative method converge strongly to a common solution of split variational inclusion problem and fixed point problem for a nonexpansive mapping which is the unique solution of the variational inequality problem. The results presented in this paper are the supplement, extension and generalization of the previously known results in this area.

Abstract: To eliminate the influence of target returns on the estimation of local sea clutter distributions, an improved iterative censoring scheme (ICS) for constant false-alarm rate detectors is proposed with two modifications. First, the proposed ICS censors out both target pixels and their four-connected neighborhood pixels from the estimation of local sea clutter distributions. Second, a novel initial detector is proposed to improve the convergence speed of ICS. The proposed initial detector, which only needs the probability of false alarms as an input parameter, is based on the sorting of all pixels under test. Experiments of ship detection with RADARSAT-2 ScanSAR wide mode images are presented to illustrate the effectiveness and improvements of the proposed ICS.

Abstract: This paper is concerned with a new iterative theta-adaptive dynamic programming (ADP) technique to solve optimal control problems of infinite horizon discrete-time nonlinear systems. The idea is to use an iterative ADP algorithm to obtain the iterative control law which optimizes the iterative performance index function. In the present iterative theta-ADP algorithm, the condition of initial admissible control in policy iteration algorithm is avoided. It is proved that all the iterative controls obtained in the iterative theta-ADP algorithm can stabilize the nonlinear system which means that the iterative theta-ADP algorithm is feasible for implementations both online and offline. Convergence analysis of the performance index function is presented to guarantee that the iterative performance index function will converge to the optimum monotonically. Neural networks are used to approximate the performance index function and compute the optimal control policy, respectively, for facilitating the implementation of the iterative theta-ADP algorithm. Finally, two simulation examples are given to illustrate the performance of the established method.

Abstract: This paper deals with the important topic of industrial robot identification. The usual identification method is based on the inverse dynamic identification model and the least squares technique. This method has been successfully applied on several industrial robots. Good results can be obtained, provided a well tuned derivative band-pass filtering of joint positions is used to calculate the joint velocities and accelerations. However, one cannot be sure whether or not the band-pass filtering is well tuned. An alternative is the instrumental variable (IV) method, which is robust to data filtering and is statistically optimal. In this paper, a generic IV approach suitable for robot identification is proposed. The instrument set is the inverse dynamic model built from simulated data calculated from simulation of the direct dynamic model. The simulation is based on previous estimates and assumes the same reference trajectories and the same control structure for both actual and simulated robots. Finally, gains of the simulated controller are updated according to IV estimates to obtain a valid instrument set at each step of the algorithm. The proposed approach validates the inverse and direct dynamic models simultaneously, is not sensitive to initial conditions, and converges rapidly. Experimental results obtained on a six-degrees-of-freedom industrial robot show the effectiveness of this approach: 60 dynamic parameters are identified in three iterations.

Abstract: In this paper, we present an extension of the iterative closest point (ICP) algorithm that simultaneously registers multiple 3D scans. While ICP fails to utilize the multiview constraints available, our method exploits the information redundancy in a set of 3D scans by using the averaging of relative motions. This averaging method utilizes the Lie group structure of motions, resulting in a 3D registration method that is both efficient and accurate. In addition, we present two variants of our approach, i.e., a method that solves for multiview 3D registration while obeying causality and a transitive correspondence variant that efficiently solves the correspondence problem across multiple scans. We present experimental results to characterize our method and explain its behavior as well as those of some other multiview registration methods in the literature. We establish the superior accuracy of our method in comparison to these multiview methods with registration results on a set of well-known real datasets of 3D scans.

Abstract: This paper studies the problem of determining the sensor locations in a large sensor network using only relative distance (range) measurements. Based on a generalized barycentric coordinate representation, our work generalizes the DILOC algorithm to the localization problem under arbitrary deployments of sensor nodes and anchor nodes. First, a criterion and algorithm are developed to determine a generalized barycentric coordinate of a node with respect to its neighboring nodes, which do not require the node to be inside the convex hull of its neighbors. Next, for the localization problem based on the generalized barycentric coordinate representation, a necessary and sufficient condition for the localizability of a sensor network with a generic configuration is obtained. Finally, a new linear iterative algorithm is proposed to ensure distributed implementation as well as global convergence to the true coordinates.