Two algorithms for minimizing a sum of squares are described and com pared. The first one is the well-known Gauss-Newton algorithm. The second one is based on the algorithm given by Marquardt.

Nonlinear least squares estimation

An Algorithm for Least-Squares Estimation of Nonlinear Parameters

The standard method for solving least squares problems which lead to non-linear normal equations depends upon a reduction of the residuals to linear form by first order Taylor approximations taken about an initial or trial solution for the parameters.2 If the usual least squares procedure, performed with these linear approximations, yields new values for the parameters which are not sufficiently close to the initial values, the neglect of second and higher order terms may invalidate the process, and may actually give rise to a larger value of the sum of the squares of the residuals than that corresponding to the initial solution. This failure of the standard method to improve the initial solution has received some notice in statistical applications of least squares3 and has been encountered rather frequently in connection with certain engineering applications involving the approximate representation of one function by another. The purpose of this article is to show how the problem may be solved by an extension of the standard method which insures improvement of the initial solution.4 The process can also be used for solving non-linear simultaneous equations, in which case it may be considered an extension of Newton's method. Let the function to be approximated be h{x, y, z, • • • ), and let the approximating function be H{oc, y, z, • • ■ ; a, j3, y, ■ • ■ ), where a, /3, 7, • ■ ■ are the unknown parameters. Then the residuals at the points, yit zit • • • ), i = 1, 2, ■ • • , n, are

/pdf/a-method-for-the-solution-of-certain-non-linear-problems-in-4jzfz2kssg.pdf

A method for the solution of certain non – linear problems in least squares

© The British Computer Society Issue Section: Articles Download all figures A powerful iterative descent method for finding a local minimum of a function of several variables is described. A number of theorems are proved to show that it always converges and that it converges rapidly. Numerical tests on a variety of functions confirm these theorems. The method has been used to solve a system of one hundred non-linear simultaneous equations. Related articles in Web of Science

/pdf/a-rapidly-convergent-descent-method-for-minimization-3urmn4uavw.pdf

A Rapidly Convergent Descent Method for Minimization

A new approach to variable metric algorithms

For given data ($t_i\ , y_i), i=1, \ldots ,m$ , we consider the least squares fit of nonlinear models of the form F($\underset ~\to a\ , \underset ~\to \alpha\ ; t) = \sum_{j=1}^{n}\ g_j (\underset ~\to a ) \varphi_j (\underset ~\to \alpha\ ; t) , \underset ~\to a\ \epsilon R^s\ , \underset ~\to \alpha\ \epsilon R^k\ $ For this purpose we study the minimization of the nonlinear functional r($\underset ~\to a\ , \underset ~\to \alpha ) = \sum_{i=1}^{m} {(y_i - F(\underset ~\to a , \underset ~\to \alpha , t_i))}^2$ It is shown that by defining the matrix ${ \{\Phi (\underset ~\to \alpha\} }_{i,j} = \varphi_j (\underset ~\to \alpha ; t_i)$ , and the modified functional $r_2(\underset ~\to \alpha ) = \l\ \underset ~\to y\ - \Phi (\underset ~\to \alpha )\Phi^+(\underset ~\to \alpha ) \underset ~\to y \l_2^2$, it is possible to optimize first with respect to the parameters $\underset ~\to \alpha$ , and then to obtain, a posteriori, the optimal parameters $\overset ^\to {\underset ~\to a}$ The matrix $\Phi^+(\underset ~\to \alpha$) is the Moore-Penrose generalized inverse of $\Phi (\underset ~\to \alpha$), and we develop formulas for its Frechet derivative under the hypothesis that $\Phi (\underset ~\to \alpha$) is of constant (though not necessarily full) rank From these formulas we readily obtain the derivatives of the orthogonal projectors associated with $\Phi (\underset ~\to \alpha$), and also that of the functional $r_2(\underset ~\to \alpha$) Detailed algorithms are presented which make extensive use of well-known reliable linear least squares techniques, and numerical results and comparisons are given These results are generalizations of those of H D Scolnik [1971]

The differentiation of pseudoinverses and nonlinear least squares problems whose variables separate.

Handle everyday research tasks with reliable, citation-backed results

Your personal Research Agent to handle research tasks with citation-backed results

Popular Tasks used by Researchers

How can I help with your research?

Meet SciSpace

Get more enhanced response by uploading the PDFs you want me to reference.

No relevant PDFs in your library

SciSpace is the AI research assistant for academics. Run systematic literature reviews on 280M+ papers, and write papers with cited sources. Trusted by 1M+ students, PhDs & researchers.

SciSpace | AI for Research

Analyze PDFs

Code & Manuscripts

Funding & Grants

Literature & Patents

Medical & Clinical Data

Systematic Review

Visualize & Present

Web & Data

Build a Google Scholar-like website for your research.

Build a website

Create charts and images for your research

Create a Chart

Write a paper for submission to a journal

Draft a manuscript

Patent Search

Design eye-catching scientific posters in minutes.

Scientific Poster Generation

Systematic Literature Review

One task is running at the moment. Your messages will be shown right after.

Drag and drop or click here to browse

Loved by <highlight>1 million+</highlight> researchers

Extract a list of specific topics and their sources from unstructured text

Topics

Compare and analyze relevant papers that matches with your search

Papers

Get insights from PDFs and bookmarked papers from your library

My library

Recent searches

Try searching for:

Catch AI-generated content in scholarly and non-scholarly content

{ai} Detector

Ai Writer

Get PDF Summaries, highlighted text explanations 

Chat with PDF

Effortlessly create in-text citations and bibliographies in APA and 2,500 other formats

Citation generator

Get explanations, summaries, and answers on academic papers

Ease up your research workflow with {scispace}'s cohort of exciting AI tools

Elevate your academic writing skills and convey your ideas the way you want

Paraphraser

Explore our range of reading and writing tools

Your file is being prepared and should be ready in a few minutes. If it's a large file, it might take a bit longer. You can close this window, and we'll email you the file when it's done.

You have reached a maximum limit of <strong>{limit}</strong> columns in the table. Remove at least <strong>1</strong> column to add or create another one.

Nonlinear least squares estimation

Chat with Paper

AI Agents for this Paper

References

An Algorithm for Least-Squares Estimation of Nonlinear Parameters

A method for the solution of certain non – linear problems in least squares

A Rapidly Convergent Descent Method for Minimization

A new approach to variable metric algorithms

The differentiation of pseudoinverses and nonlinear least squares problems whose variables separate.