Abstract: We present a technique for constructing random fields from a set of training samples. The learning paradigm builds increasingly complex fields by allowing potential functions, or features, that are supported by increasingly large subgraphs. Each feature has a weight that is trained by minimizing the Kullback-Leibler divergence between the model and the empirical distribution of the training data. A greedy algorithm determines how features are incrementally added to the field and an iterative scaling algorithm is used to estimate the optimal values of the weights. The random field models and techniques introduced in this paper differ from those common to much of the computer vision literature in that the underlying random fields are non-Markovian and have a large number of parameters that must be estimated. Relations to other learning approaches, including decision trees, are given. As a demonstration of the method, we describe its application to the problem of automatic word classification in natural language processing.

Abstract: Krylov subspace methods for approximating the action of matrix exponentials are analyzed in this paper. We derive error bounds via a functional calculus of Arnoldi and Lanczos methods that reduces the study of Krylov subspace approximations of functions of matrices to that of linear systems of equations. As a side result, we obtain error bounds for Galerkin-type Krylov methods for linear equations, namely, the biconjugate gradient method and the full orthogonalization method. For Krylov approximations to matrix exponentials, we show superlinear error decay from relatively small iteration numbers onwards, depending on the geometry of the numerical range, the spectrum, or the pseudospectrum. The convergence to exp$(\tau A)v$ is faster than that of corresponding Krylov methods for the solution of linear equations $(I-\tau A)x=v$, which usually arise in the numerical solution of stiff ordinary differential equations (ODEs). We therefore propose a new class of time integration methods for large systems of nonlinear differential equations which use Krylov approximations to the exponential function of the Jacobian instead of solving linear or nonlinear systems of equations in every time step.

Abstract: Foreword Preface Glossary of Symbols 1. Introduction Part I Theory 2. Generalized Distances and Generalized Projections 3. Proximal Minimization with D-Functions Part II Algorithms 4. Penalty Methods, Barrier Methods and Augmented Lagrangians 5. Iterative Methods for Convex Feasibility Problems 6. Iterative Algorithms for Linearly Constrained Optimization Problems 7. Model Decomposition Algorithms 8. Decompositions in Interior Point Algorithms Part III Applications 9. Matrix Estimation Problems 10. Image Reconsturction from Projections 11. The Inverse Problem in Radiation Therapy Treatment Planning 12. Multicommodity Network Flow Problems 13. Planning Under Uncertainty 14. Decompositions for Parallel Computing 15. Numerical Investigations

Abstract: In this paper we present a new algorithm for accelerating the potential calculation which occurs in the inner loop of iterative algorithms for solving electromagnetic boundary integral equations. Such integral equations arise, for example, in the extraction of coupling capacitances in three-dimensional (3-D) geometries. We present extensive experimental comparisons with the capacitance extraction code FASTCAP and demonstrate that, for a wide variety of geometries commonly encountered in integrated circuit packaging, on-chip interconnect and micro-electro-mechanical systems, the new "precorrected-FFT" algorithm is superior to the fast multipole algorithm used in FASTCAP in terms of execution time and memory use. At engineering accuracies, in terms of a speed-memory product, the new algorithm can be superior to the fast multipole based schemes by more than an order of magnitude.

Abstract: A parallel preconditioner is presented for the solution of general sparse linear systems of equations. A sparse approximate inverse is computed explicitly and then applied as a preconditioner to an iterative method. The computation of the preconditioner is inherently parallel, and its application only requires a matrix-vector product. The sparsity pattern of the approximate inverse is not imposed a priori but captured automatically. This keeps the amount of work and the number of nonzero entries in the preconditioner to a minimum. Rigorous bounds on the clustering of the eigenvalues and the singular values are derived for the preconditioned system, and the proximity of the approximate to the true inverse is estimated. An extensive set of test problems from scientific and industrial applications provides convincing evidence of the effectiveness of this approach.

Abstract: A simple iterative decoding technique using hard-decision feedback is presented for bit-interleaved coded modulation (BICM). With an 8-state, rate-2/3 convolutional code, and 8-PSK modulation, the improvement over the conventional BICM scheme exceeds 1 dB for a fully-interleaved Rayleigh flat-fading channel and exceeds 1.5 dB for a channel with additive white Gaussian noise. This robust performance makes BICM with iterative decoding suitable for both types of channels.

Abstract: Originated from the practical implementation and numerical considerations of iterative methods for solving mathematical programs, the study of error bounds has grown and proliferated in many interesting areas within mathematical programming. This paper gives a comprehensive, state-of-the-art survey of the extensive theory and rich applications of error bounds for inequality and optimization systems and solution sets of equilibrium problems.

Abstract: A review has been given of the surface-related multiple problem by making use of the so-called feedback model. From the resulting equations it has been concluded that the proposed solution does not require any properties of the subsurface. However, source-detector and reflectivity properties of the surface need be specified. Those properties have been quantified in a surface operator and this operator is estimated as part of the multiple removal problem. The surface-related multiple removal algorithm has been formulated in terms of a Neumann series and in terms of an iterative equation. The Neumann formulation requires a nonlinear optimization process for the surface operator; while the iterative formulation needs a number of linear optimizations. The iterative formulation also has the advantage that it can be integrated easily with another multiple removal method. An algorithm for the removal of internal multiples has been proposed as well. This algorithm is an extension of the surface-related method. Removal of internal multiples requires knowledge of the macro velocity model between the surface and the upper boundary of the multiple generating layer. In Part II (also published in this issue) the success of the proposed algorithms has been demonstrated on numerical experiments and field data examples.

Abstract: Various methods for efficiently solving electromagnetic problems are presented. Electromagnetic scattering problems can be roughly classified into surface and volume problems, while fast methods are either differential or integral equation based. The resultant systems of linear equations are either solved directly or iteratively. A review of various differential equation solvers, their complexities, and memory requirements is given. The issues of grid dispersion and hybridization with integral equation solvers are discussed. Several fast integral equation solvers for surface and volume scatterers are presented. These solvers have reduced computational complexities and memory requirements.

Abstract: This paper refers to quantitative reconstruction of the dielectric and conductive property distributions of biological objects by means of active microwave imaging. An iterative reconstruction algorithm based on the Levenberg-Marquardt method is tested with synthetic data. The influence of the receiver geometry is investigated and two methods for choosing the regularization parameter, an empirical method and generalized cross validation (GCV) method, are examined.

Abstract: A surface-related multiple-elimination method can be formulated as an iterative procedure: the output of one iteration step is used as input for the next iteration step (part I of this paper). In this paper (part II) it is shown that the procedure can be made very efficient if a good initial estimate of the multiple-free data set can be provided in the first iteration, and in many situations, the Radon-based multiple-elimination method may provide such an estimate. It is also shown that for each iteration, the inverse source wavelet can be accurately estimated by a linear (least-squares) inversion process. Optionally, source and detector variations and directivity effects can be included, although the examples are given without these options. The iterative multiple elimination process, together with the source wavelet estimation, are illustrated with numerical experiments as well as with field data examples. The results show that the surface-related multiple-elimination process is very effective in time gates where the moveout properties of primaries and multiples are very similar (generally deep data), as well as for situations with a complex multiple-generating system.

Abstract: The problem of determining the proper size of an artificial neural network is recognized to be crucial, especially for its practical implications in such important issues as learning and generalization. One popular approach for tackling this problem is commonly known as pruning and it consists of training a larger than necessary network and then removing unnecessary weights/nodes. In this paper, a new pruning method is developed, based on the idea of iteratively eliminating units and adjusting the remaining weights in such a way that the network performance does not worsen over the entire training set. The pruning problem is formulated in terms of solving a system of linear equations, and a very efficient conjugate gradient algorithm is used for solving it, in the least-squares sense. The algorithm also provides a simple criterion for choosing the units to be removed, which has proved to work well in practice. The results obtained over various test problems demonstrate the effectiveness of the proposed approach.

Abstract: In this paper we prove that the iteratively regularized Gauss-Newton method is a locally convergent method for solving nonlinear ill-posed problems, provided the nonlinear operator satisfies a certain smoothness condition. For perturbed data we propose a priori and a posteriori stopping rules that guarantee convergence of the iterates, if the noise level goes to zero. Under appropriate closeness and smoothness conditions on the exact solution we obtain the same convergence rates as for linear ill-posed problems.

Abstract: The expectation-maximization (EM) algorithm was first introduced in the statistics literature as an iterative procedure that under some conditions produces maximum-likelihood (hit) parameter estimates. In this paper we investigate the application of the EM algorithm to sequence estimation in the presence of random disturbances and additive white Gaussian noise. As examples of the use of the EM algorithm, we look at the random-phase and fading channels, and show that a formulation of the sequence estimation problem based on the EM algorithm can provide a means of obtaining ML sequence estimates, a task that has been previously too complex to perform.

Abstract: In total variation denoising, one attempts to remove noise from a signal or image by solving a nonlinear minimization problem involving a total variation criterion. Several approaches based on this idea have recently been shown to be very effective, particularly for denoising functions with discontinuities. This paper analyzes the convergence of an iterative method for solving such problems. The iterative method involves a "lagged diffusivity" approach in which a sequence of linear diffusion problems are solved. Global convergence in a finite-dimensional setting is established, and local convergence properties, including rates and their dependence on various parameters, are examined.

Abstract: We present an efficient algorithm for estimating the time delays and the directions-of-arrival (DOAs) of multiple reflections of a known signal. The algorithm is based on an iterative scheme that transforms the multidimensional maximum likelihood criterion into two sets of simple one-dimensional (1-D) maximization problems. Simulation results illustrating the performance of the algorithm in comparison with the Cramer-Rao bound are included.

Abstract: Optical tomography is a new medical imaging modality that is at the threshold of realization. A large amount of clinical work has shown the very real benefits that such a method could provide. At the same time a considerable effort has been put into theoretical studies of its probable success. At present there exist gaps between these two realms. In this paper we review some general approaches to inverse problems to set the context for optical tomography, defining both the terms forward problem and inverse problem. An essential requirement is to treat the problem in a nonlinear fashion, by using an iterative method. This in turn requires a convenient method of evaluating the forward problem, and its derivatives and variance. Photon transport models are described for obtaining analytical and numerical solutions for the most commonly used ones are reviewed. The inverse problem is approached by classical gradient-based solution methods. In order to develop practical implementations of these methods, we discuss the important topic of photon measurement density functions, which represent the derivative of the forward problem. We show some results that represent the most complex and realistic simulations of optical tomography yet developed. We suggest, in particular, that both time-resolved, and intensity-modulated systems can reconstruct variations in both optical absorption and scattering, but that unmodulated, non-time-resolved systems are prone to severe artefact. We believe that optical tomography reconstruction methods can now be reliably applied to a wide variety of real clinical data. The expected resolution of the method is poor, meaning that it is unlikely that the type of high-resolution images seen in computed tomography or medical resonance imaging can ever be obtained. Nevertheless we strongly expect the functional nature of these images to have a high degree of clinical significance.

Abstract: A new method for energy loss reduction in distribution networks is presented It is based on known techniques and algorithms for radial network analysis-oriented element ordering, power summation method for power flow, statistical representation of load variations and a recently developed energy summation method for the computation of energy losses These methods, combined with the heuristic rules developed to lead the iterative process, make the energy loss minimization method effective, robust and fast It presents an alternative to the power minimization methods for operation and planning purposes

Abstract: In this paper, a time-varying pricing model of a road bottleneck with elastic traffic demand is formulated using the optimal control theory. It is assumed that the optimal use of the bottleneck is achieved when social benefit over the whole time horizon of study is maximized. The necessary conditions for optimal solution are derived and their economic interpretations are given. Different from conventional analyses, queuing is not pre-assumed to be zero when obtaining the optimal time-varying toll, and the exit capacity of the bottleneck is assumed either to be constant or to vary with queue length. An approximate iterative algorithm is proposed for solving the model in a discrete time version. Three numerical examples are presented to demonstrate the applications of the proposed model and algorithm.

Abstract: The projected Landweber method is an iterative method for solving constrained least-squares problems when the constraints are expressed in terms of a convex and closed set . The convergence properties of the method have been recently investigated. Moreover, it has important applications to many problems of signal processing and image restoration. The practical difficulty is that the convergence is too slow. In this paper we apply to this method the so-called preconditioning which is frequently used for increasing the efficiency of the conjugate gradient method. We discuss the significance of preconditioning in this case and we show that it implies a modification of the original constrained least-squares problem. However, when the original problem is ill-posed, the approximate solutions provided by the preconditioned method are similar to those provided by the standard method if the preconditioning is suitably chosen. Moreover, the number of iterations can be reduced by a factor of 10 and even more. A few applications to problems of image restoration are also discussed.

Abstract: Several vision problems can be reduced to the problem of fitting a linear surface of low dimension to data, including the problems of structure-from-affine-motion, and of characterizing the intensity images of a Lambertian scene by constructing the intensity manifold. For these problems, one must deal with a data matrix with some missing elements. In structure-from-motion, missing elements will occur if some point features are not visible in some frames. To construct the intensity manifold missing matrix elements will arise when the surface normals of some scene points do not face the light source in some images. We propose a novel method for fitting a low rank matrix to a matrix with missing elements. We show experimentally that our method produces good results in the presence of noise. These results can be either used directly, or can serve as an excellent starting point for an iterative method.

Abstract: This paper presents a general unified hybrid method for radiation and scattering problems such as antennas mounted on a large platform. The method uses a coupled electric-field integral equation (EFIE) and magnetic-field integral equation (MFIE) formulation, referred to as the hybrid EFIE-MFIE (HEM), in which the EFIE and MFIE are applied to geometrically distinct regions of an object. The HEM is capable of modeling arbitrary three-dimensional (3-D) metallic structures, including wires and both open and closed surfaces. We show that current-based hybrid techniques that utilize physical optics (PO) are an approximation of the HEM formulation. A numerical solution procedure is given that combines the moment method (EFIE) with an iterative Neumann series technique (MFIE). This permits one to effectively utilize the PO approximation when appropriate, and provides a general and systematic mechanism to correct the errors introduced by PO. Consequently, the HEM overcomes the inherent limitations of hybrid techniques which rely upon ansatz-based improvements of PO. The method is applied to the problem of radiation from objects that can be modeled using wires and metallic surfaces as fundamental elements. A representative example is given to demonstrate that the method can handle the difficult problem of a parasitic monopole located in the deep shadow region.

Abstract: This paper presents the Smoothly Constrained Kalman Filter (SCKF) for nonlinear constraints A constraint is any relation that exists between the state variables Constraints can be treated as perfect observations But, linearization errors can prevent the estimate from converging to the true value Therefore, the SCKF iteratively applies nonlinear constraints as nearly perfect observations, or, equivalently, weakened constraints Integration of new measurements is interlaced with these iterations, which reduces linearization errors and, hence, improves convergence compared to other iterative methods The weakening is achieved by artificially increasing the variance of the nonlinear constraint The paper explains how to choose the weakening values, and when to start and stop the iterative application of the constraint

Abstract: We obtain closed-form analytic solutions for surface Green`s functions within arbitrary multiorbital models. The formulation is completely general, and is equally valid for empirical tight binding, linear-muffin-tin-orbital tight binding, screened Korringa-Kohn-Rostoker and other Green`s-function equivalent formalisms, where the Hamiltonian can be put into a localized (i.e., block-band) form. The solutions are applicable to finite or semi-infinite surface systems, with quite general substrate and overlayers, or even to superlattices. This is achieved by solving Dyson`s equations by means of a matrix-valued extension of the Moebius transformation. The analytical properties of the solutions are discussed, and by considering their asymptotic limit, a simple closed form for the exact (semi-infinite) surface Green`s function is obtained. The numerical calculation of the surface Green`s function (or of observable quantities such as the density of states) using this closed form is compared with previously known iterative procedures. We find that it is far faster, far more stable, and more accurate than the best iterative method. {copyright} {ital 1997} {ital The American Physical Society}

Abstract: The problem of determining the shape and location of an object embedded in a homogeneous dissipative medium from measurements of the field scattered by the object is considered in this paper. The object is assumed to be an infinite cylinder of known cross section illuminated by a TM plane wave and the scattered field is measured on a line segment perpendicular to the direction of incidence. Measurement data are carried out at three different frequencies for a homogeneous cylinder of known dielectric constant. The location and contour shape are determined using two different reconstruction algorithms, a Newton-Kantorovich (NK) method and the modified gradient (MG) method whose effectiveness and robustness are compared. Both methods are based on domain integral representations of the field in the body. They involve an iterative minimization of the defect between an integral representation of the field measured on the line and the actual measured data. The NK method involves a linearization of the nonlinear relation between the field and the contrast, as well as the solution of a direct scattering problem at each iteration. The MG method seeks the simultaneous reconstruction of the field and the characteristic function of the support of the scatterer without solving a direct problem at each step. Both methods employed the same initial guess and the a priori information that the characteristic function is nonnegative.

Abstract: 1. Introduction 2. Generations of Wavefields Using Diffractive Optical Elements (DOEs) 3. Parametric Methods of Computing DOEs 4. Iterative Algorithms for Calculating DOEs Forming Radially Symmetrical Images 5. Iterative Algorithms for CalculatingWavefront Formers 6. Calculation of Phase Formers of Light Modes 7. Design of Multiorder Diffractive Gratings with a Pregiven Intensity of Diffractive Orders 8. Iterative Methods for Calculating Multifocus DOEs 9. Calculation of DOEs for Some SpecialApplications 10. Conclusion Appendices References Index.

Abstract: SUMMARY A non exhaustive overview of shock absorber models is presented. The ability of the models to match experimental data is emphasized. Two physical models are presented that are able to extract the internal valve parameters from data without hysteresis. In order to implement a model that copes with hysteresis, most models require the numerical solution to a set of nonlinear differential equations. The use of an alternative restoring force method can get round the time consuming iterative simulation and identification routines. The alternative nonparametric method models the force as a function of velocity and acceleration. The theoretical relevance of the model is studied.