Well-posed problem

Abstract: When discrete ill-posed problems are analyzed and solved by various numerical regularization techniques, a very convenient way to display information about the regularized solution is to plot the norm or seminorm of the solution versus the norm of the residual vector. In particular, the graph associated with Tikhonov regularization plays a central role. The main purpose of this paper is to advocate the use of this graph in the numerical treatment of discrete ill-posed problems. The graph is characterized quantitatively, and several important relations between regularized solutions and the graph are derived. It is also demonstrated that several methods for choosing the regularization parameter are related to locating a characteristic L-shaped “corner” of the graph.

Abstract: Regularization algorithms are often used to produce reasonable solutions to ill-posed problems. The L-curve is a plot—for all valid regularization parameters—of the size of the regularized solution versus the size of the corresponding residual. Two main results are established. First a unifying characterization of various regularization methods is given and it is shown that the measurement of “size” is dependent on the particular regularization method chosen. For example, the 2-norm is appropriate for Tikhonov regularization, but a 1-norm in the coordinate system of the singular value decomposition (SVD) is relevant to truncated SVD regularization. Second, a new method is proposed for choosing the regularization parameter based on the L-curve, and it is shown how this method can be implemented efficiently. The method is compared to generalized cross validation and this new method is shown to be more robust in the presence of correlated errors.

Abstract: Preface Symbols and Acronyms 1. Setting the Stage. Problems With Ill-Conditioned Matrices Ill-Posed and Inverse Problems Prelude to Regularization Four Test Problems 2. Decompositions and Other Tools. The SVD and its Generalizations Rank-Revealing Decompositions Transformation to Standard Form Computation of the SVE 3. Methods for Rank-Deficient Problems. Numerical Rank Truncated SVD and GSVD Truncated Rank-Revealing Decompositions Truncated Decompositions in Action 4. Problems with Ill-Determined Rank. Characteristics of Discrete Ill-Posed Problems Filter Factors Working with Seminorms The Resolution Matrix, Bias, and Variance The Discrete Picard Condition L-Curve Analysis Random Test Matrices for Regularization Methods The Analysis Tools in Action 5. Direct Regularization Methods. Tikhonov Regularization The Regularized General Gauss-Markov Linear Model Truncated SVD and GSVD Again Algorithms Based on Total Least Squares Mollifier Methods Other Direct Methods Characterization of Regularization Methods Direct Regularization Methods in Action 6. Iterative Regularization Methods. Some Practicalities Classical Stationary Iterative Methods Regularizing CG Iterations Convergence Properties of Regularizing CG Iterations The LSQR Algorithm in Finite Precision Hybrid Methods Iterative Regularization Methods in Action 7. Parameter-Choice Methods. Pragmatic Parameter Choice The Discrepancy Principle Methods Based on Error Estimation Generalized Cross-Validation The L-Curve Criterion Parameter-Choice Methods in Action Experimental Comparisons of the Methods 8. Regularization Tools Bibliography Index.