Abstract: In this paper, we present an algorithm to estimate a signal from its modified short-time Fourier transform (STFT). This algorithm is computationally simple and is obtained by minimizing the mean squared error between the STFT of the estimated signal and the modified STFT. Using this algorithm, we also develop an iterative algorithm to estimate a signal from its modified STFT magnitude. The iterative algorithm is shown to decrease, in each iteration, the mean squared error between the STFT magnitude of the estimated signal and the modified STFT magnitude. The major computation involved in the iterative algorithm is the discrete Fourier transform (DFT) computation, and the algorithm appears to be real-time implementable with current hardware technology. The algorithm developed in this paper has been applied to the time-scale modification of speech. The resulting system generates very high-quality speech, and appears to be better in performance than any existing method.

Abstract: 1. Introduction.- 1.1 Function Optimization and Parameter Optimization.- 1.2 Elements of Problem Formulation.- Design Variables.- Objective Function.- Constraints.- Standard Formulation.- 1.3 The Solution Process.- 1.4 Analysis and Design Formulations.- 1.5 Specific Versus General Methods.- 1.6 Exercises.- 1.7 References.- 2. Classical Tools in Structural Optimization.- 2.1 Optimization Using Differential Calculus.- 2.2 Optimization Using Variational Calculus.- to the Calculus of Variations.- 2.3 Classical Methods for Constrained Problems.- Method of Lagrange Multipliers.- Function Subjected to an Integral Constraint.- Finite Subsidiary Conditions.- 2.4 Local Constraints and the Minmax Approach.- 2.5 Necessary and Sufficient Conditions for Optimality.- Elastic Structures of Maximum Stiffness.- Optimal Design of Euler-Bernoulli Columns.- Optimum Vibrating Euler-Bernoulli Beams.- 2.6 Use of Series Solutions in Structural Optimization.- 2.7 Exercises.- 2.8 References.- 3. Linear Programming.- 3.1 Limit Analysis and Design of Structures Formulated as LP Problems.- 3.2 Prestressed Concrete Design by Linear Programming.- 3.3 Minimum Weight Design of Statically Determinate Trusses.- 3.4 Graphical Solutions of Simple LP Problems.- 3.5 A Linear Program in a Standard Form.- Basic Solution.- 3.6 The Simplex Method.- Changing the Basis.- Improving the Objective Function.- Generating a Basic Feasible Solution-Use of Artificial Variables.- 3.7 Duality in Linear Programming.- 3.8 An Interior Method-Karmarkar's Algorithm.- Direction of Move.- Transformation of Coordinates.- Move Distance.- 3.9 Integer Linear Programming.- Branch-and-Bound Algorithm.- 3.10 Exercises.- 3.11 References.- 4. Unconstrained Optimization.- 4.1 Minimization of Functions of One Variable.- Zeroth Order Methods.- First Order Methods.- Second Order Method.- Safeguarded Polynomial Interpolation.- 4.2 Minimization of Functions of Several Variables.- Zeroth Order Methods.- First Order Methods.- Second Order Methods.- Applications to Analysis.- 4.3 Specialized Quasi-Newton Methods.- Exploiting Sparsity.- Coercion of Hessians for Suitability with Quasi-Newton Methods.- Making Quasi-Newton Methods Globally Convergent.- 4.4 Probabilistic Search Algorithms.- Simulated Annealing.- Genetic Algorithms.- 4.5 Exercises.- 4.6 References.- 5. Constrained Optimization.- 5.1 The Kuhn-Tucker Conditions.- General Case.- Convex Problems.- 5.2 Quadratic Programming Problems.- 5.3 Computing the Lagrange Multipliers.- 5.4 Sensitivity of Optimum Solution to Problem Parameters.- 5.5 Gradient Projection and Reduced Gradient Methods.- 5.6 The Feasible Directions Method.- 5.7 Penalty Function Methods.- Exterior Penalty Function.- Interior and Extended Interior Penalty Functions.- Unconstrained Minimization with Penalty Functions.- Integer Programming with Penalty Functions.- 5.8 Multiplier Methods.- 5.9 Projected Lagrangian Methods (Sequential Quadratic Prog.).- 5.10 Exercises.- 5.11 References.- 6. Aspects of the Optimization Process in Practice.- 6.1 Generic Approximations.- Local Approximations.- Global and Midrange Approximations.- 6.2 Fast Reanalysis Techniques.- Linear Static Response.- Eigenvalue Problems.- 6.3 Sequential Linear Programming.- 6.4 Sequential Nonlinear Approximate Optimization.- 6.5 Special Problems Associated with Shape Optimization.- 6.6 Optimization Packages.- 6.7 Test Problems.- Ten-Bar Truss.- Twenty-Five-Bar Truss.- Seventy-Two-Bar Truss.- 6.8 Exercises.- 6.9 References.- 7. Sensitivity of Discrete Systems.- 7.1 Finite Difference Approximations.- Accuracy and Step Size Selection.- Iterative Methods.- Effect of Derivative Magnitude on Accuracy.- 7.2 Sensitivity Derivatives of Static Displacement and Stress Constraints.- Analytical First Derivatives.- Second Derivatives.- The Semi-Analytical Method.- Nonlinear Analysis.- Sensitivity of Limit Loads.- 7.3 Sensitivity Calculations for Eigenvalue Problems.- Sensitivity Derivatives of Vibration and Buckling Constraints.- Sensitivity Derivatives for Non-Hermitian Eigenvalue Problems.- Sensitivity Derivatives for Nonlinear Eigenvalue Problems.- 7.4 Sensitivity of Constraints on Transient Response.- Equivalent Constraints.- Derivatives of Constraints.- Linear Structural Dynamics.- 7.5 Exercises.- 7.6 References.- 8. Introduction to Variational Sensitivity Analysis.- 8.1 Linear Static Analysis.- The Direct Method.- The Adjoint Method.- Implementation Notes.- 8.2 Nonlinear Static Analysis and Limit Loads.- Static Analysis.- Limit Loads.- Implementation Notes.- 8.3 Vibration and Buckling.- The Direct Method.- The Adjoint Method.- 8.4 Static Shape Sensitivity.- The Material Derivative.- Domain Parametrization.- The Direct Method.- The Adjoint Method.- 8.5 Exercise.- 8.6 References.- 9. Dual and Optimality Criteria Methods.- 9.1 Intuitive Optimality Criteria Methods.- Fully Stressed Design.- Other Intuitive Methods.- 9.2 Dual Methods.- General Formulation.- Application to Separable Problems.- Discrete Design Variables.- Application with First Order Approximations.- 9.3 Optimality Criteria Methods for a Single Constraint.- The Reciprocal Approximation for a Displacement Constraint.- A Single Displacement Constraint.- Generalization for Other Constraints.- Scaling-based Resizing.- 9.4 Several Constraints.- Reciprocal-Approximation Based Approach.- Scaling-based Approach.- Other Formulations.- 9.5 Exercises.- 9.6 References.- 10. Decomposition and Multilevel Optimization.- 10.1 The Relation between Decomposition and Multilevel Formulation.- 10.2 Decomposition.- 10.3 Coordination and Multilevel Optimization.- 10.4 Penalty and Envelope Function Approaches.- 10.5 Narrow-Tree Multilevel Problems.- Simultaneous Analysis and Design.- Other Applications.- 10.6 Decomposition in Response and Sensitivity Calculations.- 10.7 Exercises.- 10.8 References.- 11.Optimum Design of Laminated Composite Materials.- 11.1 Mechanical Response of a Laminate.- Orthotropic Lamina.- Classical Laminated Plate Theory.- Bending, Extension, and Shear Coupling.- 11.2 Laminate Design.- Design of Laminates for In-plane Response.- Design of Laminates for Flexural Response.- 11.3 Stacking Sequence Design.- Graphical Stacking Sequence Design.- Penalty Function Formulation.- Integer Linear Programming Formulation.- Probabilistic Search Methods.- 11.4 Design Applications.- Stiffened Plate Design.- Aeroelastic Tailoring.- 11.5 Design Uncertainties.- 11.6 Exercises.- 11.7 References.- Name Index.

Abstract: The classical optimal power flow problem with a nonseparable objective function can be solved by an explicit Newton approach. Efficient, robust solutions can be obtained for problems of any practical size or kind. Solution effort is approximately proportional to network size, and is relatively independent of the number of controls or binding inequalities. The key idea is a direct simultaneous solution for all of the unknowns in the Lagrangian function on each iteration. Each iteration minimizes a quadratic approximation of the Lagrangian. For any given set of binding constraints the process converges to the Kuhn-Tucker conditions in a few iterations. The challenge in algorithm development is to efficiently identify the binding inequalities.

Abstract: Geophysical inversion involves the estimation of the parameters of a postulated earth model from a set of observations. Since the associated model responses can be nonlinear functions of the model parameters, nonlinear least-squares techniques prove to be useful for performing the inversion. A common type of inversion applies iterative damped linear least squares through use of the Marquardt-Levenberg method. Traditionally, this method has been implemented by solving the associated normal equations in conventional ways. However, Singular Value Decomposition (SVD) produces significant improvements in computational precision when applied to the same system of normal equations. Iterative least-squares modeling finds application in a wide variety of geophysical problems. Two examples illustrate the approach: (1) seismic wavelet deconvolution, and (2) the location of a buried wedge from surface gravity data. More generally, nonlinear least-squares inversion can be used to estimate earth models for any set of geophysical observations for which an appropriate mathematical description is available.

Abstract: This is the first of a series of papers giving the solution of the inverse problem in seismic exploration. The acoustic approximation is used together with the assumption that the velocity field has the form . The forward problem is then linearized (thus neglecting multiple reflected waves) and the inverse problem of estimating δ is set up. Its rigorous solution can be obtained using an iterative algorithm, each step consisting of a classical Kirchhoff migration (hyperbola summation) plus a classical forward modeling step (circle summation).

Abstract: In the presence of several user categories or transportation modes, or when transportation costs on each arc of a network depend on the flows on adjacent arcs, the traffic equilibrium problem may be expressed as a variational problem. Methods for determining traffic equilibria are then adaptations of techniques for solving variational inequalities. In this paper, we present a new convex optimization formulation for the general traffic equilibrium problem, and propose a simple iterative method for calculating traffic equilibria, which essentially involves postoptimizing a linear sub-problem at each iteration. Preliminary computational results are reported.

Abstract: An iterative technique is developed to rigorously compute the electromagnetic time- and frequency-domain scattering problems. The method is based upon a wave-function expansion technique (this also includes the integral-representation techniques), in which the electromagnetic field equations and causality conditions are satisfied analytically, while the boundary conditions or the constitutive relations have to be satisfied in a computational manner. The latter is accomplished by an iterative minimization of the integrated square error. For the solution of an integral equation, it is shown how to obtain optimum convergence. Some numerical results pertaining to a number of representative problems illustrate the numerical advantages and disadvantages of the iterative method.

Abstract: We propose an algorithm for implementing Newton's method for a general nonlinear system $f(x) = 0$ where the linear systems that arise at each step of Newton's method are solved by a preconditioned Krylov subspace iterative method. The algorithm requires only function evaluations and does not require the evaluation or storage of the Jacobian matrix. Matrix-vector products involving the Jacobian matrix are approximated by directional differences. We develop a framework for constructing preconditionings for this inner iterative method which do not reference the Jacobian matrix explicitly. We derive a nonlinear SSOR type preconditioning which numerical experiments show to be as effective as the linear SSOR preconditioning that uses the Jacobian explicitly.

Abstract: Tomosynthetic reconstructions suffer from the disadvantage that blurred images of object detail lying outside the plane of interest are superimposed over the desired image of structures in the tomosynthetic plane. It is proposed to selectively reduce these undesired superimpositions by a constrained iterative restoration method, suitably generalized to permit simultaneous deconvolution of multiple planes. Sufficient conditions are derived ensuring the convergence of the iterations to the exact solution in the absence of noise and constraints. Although in practice the restoration process must be left incomplete because of inescapable noise and quantization artifacts, the experimental results demonstrate that for reasons of stability the convergence conditions derived for the noise-free, unconstrained case should be satisfied. In order to establish a basis for a formal stopping criterion of the iteration procedure, the buildup of noise in the sequence of iterative restorations arising from white noise in the original radiographs is investigated theoretically and experimentally. This results in the derivation of an approximation to the limiting noise variance in the reconstructions which is verified experimentally.

Abstract: An iterative segmentation method is presented and illustrated on specific examples. Full control of each iteration step is obtained by combining local and global properties according to a model of the image structure. A consistent convergence criterion is derived from additional image structure properties and a test is proposed to evaluate adequacy of segmentation.

Abstract: A fast iterative method for the solution of large, sparse, symmetric, positive definite linear complementarity problems is presented. The iterations reduce to linear iterations in a neighborhood of the solution if the problem is nondegenerate. The variational setting of the method guarantees global convergence.

Abstract: The open-loop Stackelberg game is conceptually extended to p players by the multilevel programming problem (MLPP) and can thus be used as a model for a variety of hierarchical systems in which sequential planning is the norm. The rational reaction sets for each of the players is first developed, and then the geometric properties of the linear MLPP are stated. Next, first-order necessary conditions are derived, and the problem is recast as a standard nonlinear program. A cutting plane algorithm using a vertex search procedure at each iteration is proposed to solve the linear three-level case. An example is given to highlight the results, along with some computational experience.

Abstract: The method of conjugate gradients is applied to the analysis of radiation from thin-wire antennas. With this iterative technique, it is possible to solve electrically large arbitrarily oriented wire structures without storing any matrices as is conventionally done in the method of moments. The basic difference between the proposed method and Galerkin's method, for the same expansion functions, is that for the iterative technique we are solving a least squares problem. Hence, as the order of the approximation is increased, the proposed technique guarantees a monotonic decrease of the least squared error ( \parallel AI - Y\parallel^{2} ), whereas Galerkin's method does not. Even though the method converges for any initial guess, a good one may significantly reduce the time of computation. Also, explicit error formulas are given for the rate of convergence of this method. Hence, any problem can be solved to a prespecified degree of accuracy. It is shown that the method has the advantage of a direct solution as the final solution is obtained in a finite number of steps. The method is also suitable for solving singular operator equations in which case the method monotonically converges to the least squares solution with minimum norm. Numerical results are presented for the thin-wire antennas and are compared with the solution obtained by the method of moments.

Abstract: A method is given for the solution of linear equations arising in the finite element method applied to a general elliptic problem. This method reduces the original problem to several subproblems (of the same form) considered on subregions, and an auxiliary problem. Very efficient iterative methods with the preconditioning operator and using FFT are developed for the auxiliary problem.

Abstract: Large sparse least squares problems arise in many applications, including geodetic network adjustments and finite element structural analysis. Although geodesists and engineers have been solving such problems for years, it is only relatively recently that numerical analysts have turned attention to them. In this paper we present a survey of numerical methods for large sparse linear least squares problems, focusing mainly on developments since the last comprehensive surveys of the subject published in 1976. We consider direct methods based on elimination and on orthogonalization, as well as various iterative methods. The ramifications of rank deficiency, constraints, and updating are also discussed.

Abstract: Necessary and sufficient conditions are established for the convergence of various iterative methods for solving the linear complementarity problem. The fundamental tool used is the classical notion of matrix splitting in numerical analysis. The results derived are similar to some well-known theorems on the convergence of iterative methods for square systems of linear equations. An application of the results to a strictly convex quadratic program is also given.

Abstract: Two strategies for assessing item bias are discussed: methods that compare (transformed) item difficulties unconditional on ability level and methods that compare the probabilities of correct response conditional on ability level. In the present study, the logit model was used to compare the probabilities of correct response to an item by members of two groups, these probabilities being conditional on the observed score. Here the observed score serves as an indicator of ability level. The logit model was iteratively applied: In the Tth iteration, the T items with the highest value of the bias statistic are excluded from the test, and the observed score indicator of ability for the (T + 1)th iteration is computed from the remaining items. This method was applied to simulated data. The results suggest that the iterative logit method is a substantial improvement on the noniterative one, and that the iterative method is very efficient in detecting biased and unbiased items.

Abstract: A new iterative approach is outlined for multidimensional computational analysis of two-fluid flow. Parametric surveys are described to illustrate that the method rationally predicts separation of two fluid flows under gravitational and centrifugal influences. A comparison is made between behavior computed by the method, and results reported in an experimental study of air and water flowing in elbows and pipes. 25 references, 4 figures.

Abstract: A numerical solution method of Laplace’s equation with cylindrical symmetry and mixed boundary conditions along the Z coordinate is presented. The method is based on an iterative process. It is applied to the evaluation of current density distribution in the region surrounding electrodes used for intracerebral electrical stimulations. The procedure converges quickly and after only twelve iterations the boundary conditions are satisfied within an accuracy of 0.1%. The convergence criterion is discussed and the results obtained on the current density distribution are presented.

Abstract: : In scientific computation, there is often need for the derivatives as well as the values of functions defined by computer programs. Here, it is shown how automatic differentation can be carried out in a modern computer language which permits user-defined operators and data types. The specific language used is PASCAL-SC, and differentation is implemented for variables of type GRADIENT vector of first partial derivatives with respect to the independent variables. Calculations of the results of operators or functions applied to GRADIENT variables are carried out according to the well-known rules for evaluation and differentiation of sums, differences, products, and so on. Since the differentiation is performed at compile time, the code produced is comparable in compactness and execution time to that obtained if numerical approximations are used for derivatives, and the theoretical and practical problems associated with numerical differentiation are avoided. PASCAL-SC source code is given for the necessary operators and standard functions, and it is shown how to prepare code for arbitrary differentiable functions to add to the library if desired. The effectiveness of the use of type GRADIENT is shown by an example of the solution of a system of nonlinear equations by Newton's method. (Author)

Abstract: The newly developed iterative extended boundary condition method (IEBCM) is extended to calculate the scattering by low-loss or lossless elongated dielectric objects. Specifically the iterative procedure is modified so as to utilize an initial assumption of the surface fields obtained from the Mie solution of a spherical object of the same dielectric properties. The solution for the elongated object is obtained by iteratively utilizing the regular EBCM technique to solve for objects of intermediate geometries between the substitute sphere and the object of interest. The other feature of the IEBCM which is related to subdividing the internal volume of the object into several overlapping subregions in each of which a separate field expansion is used is also utilized in the present extension. Results illustrating the adequacy of the IEBCM procedure to calculate the scattering by spheroids of an aspect ratio of more than 7:1 are presented.

Abstract: This paper presents an iterative, Newton-type method for solving a class of linear complementarity problems. This class was discovered by Mangasarian who had established that these problems can be solved as linear programs. Cottle and Pang characterized solutions of the problems in terms of least elements of certain polyhedral sets. The algorithms developed in this paper are shown to converge to the least element solutions. Some applications and computational results are also discussed.