LINCOA

Abstract: During the last ten years, the speaker has developed Fortran software for minimizing general objective functions when first derivatives are not available. He has released two packages, namely NEWUOA and UOBYQA, for the unconstrained case and for simple upper and lower bounds on the variables, respectively. His current research addresses general linear constraints on the variables, which is intended to provide the new LINCOA package. These algorithms require only of magnitude n squared operations on a typical iteration, where n is the number of variables, which has allowed them to be applied when n is in the hundreds. No attention is given to sparsity.Some excellent numerical results have been achieved, which certainly depend on the use of the symmetric Broyden updating formula. That formula when derivatives are not available is the subject of Lecture 1. It supplies the solution to the following variational problem. Some values of the objective function and a quadratic approximation to the objective function are given, where usually the approximation does not interpolate all the given function values. A new quadratic approximation is required that does interpolate the given function values. The freedom that remains in the new approximation is taken up by making as small as possible the Frobenius norm of the change to the second derivative matrix of the approximation.A trust region in the space of the variables is the set of all points that are within a prescribed distance of the best point so far. A typical change to the vector of variables is a move from the best point so far to a new point within the trust region, the new point being chosen to provide a relatively small value of the current quadratic approximation to the objective function. The calculation of the new point is the subject of Lecture 2. A version of truncated conjugate gradients is employed, in order that the amount of work is only of magnitude n2. Several decisions had to be taken to specify an algorithm. They are particularly interesting for the LINCOA software, due to the linear constraints on the variables.Lectures 1 and 2 describe vital ingredients of the software that has been mentioned. A few more details are addressed in Lecture 3, to complete an outline of how the algorithms work. The numerical results from several test problems are considered too. The accuracy and efficiency are much better than expected if one takes the view that the second derivative matrix of the quadratic approximation should become close to the second derivative matrix of the objective function. We find clearly that this view is wrong. Also the numerical results show that good accuracy is achieved eventually, although the number of iterations is sometimes very sensitive to effects from computer rounding errors.