BOBYQA

Abstract: We present two software packages for derivative-free optimization (DFO): DFO-LS for nonlinear least-squares problems and Py-BOBYQA for general objectives, both with optional bound constraints. Inspired by the Gauss-Newton method, DFO-LS constructs simplified linear regression models for the residuals and allows flexible initialization for expensive problems, whereby it can begin making progress after as few as two objective evaluations. Numerical results show DFO-LS can gain reasonable progress on some medium-scale problems with fewer objective evaluations than is needed for one gradient evaluation. DFO-LS has improved robustness to noise, allowing sample averaging, regression-based model construction, and multiple restart strategies with an auto-detection mechanism. Our extensive numerical experimentation shows that restarting the solver when stagnation is detected is a cheap and effective mechanism for achieving robustness, with superior performance over sampling and regression techniques. The package Py-BOBYQA is a Python implementation of BOBYQA (Powell 2009), with novel features such as the implementation of robustness to noise strategies. Our numerical experiments show that Py-BOBYQA is comparable to or better than existing general DFO solvers for noisy problems. In our comparisons, we introduce an adaptive accuracy measure for data profiles of noisy functions, striking a balance between measuring the true and the noisy objective improvement.

Abstract: We present DFO-LS, a software package for derivative-free optimization (DFO) for nonlinear Least-Squares (LS) problems, with optional bound constraints. Inspired by the Gauss-Newton method, DFO-LS constructs simplified linear regression models for the residuals. DFO-LS allows flexible initialization for expensive problems, whereby it can begin making progress from as few as two objective evaluations. Numerical results show DFO-LS can gain reasonable progress on some medium-scale problems with fewer objective evaluations than is needed for one gradient evaluation. DFO-LS has improved robustness to noise, allowing sample averaging, the construction of regression-based models, and multiple restart strategies together with an auto-detection mechanism. Our extensive numerical experimentation shows that restarting the solver when stagnation is detected is a cheap and effective mechanism for achieving robustness, with superior performance over both sampling and regression techniques. We also present our package Py-BOBYQA, a Python implementation of BOBYQA (Powell, 2009), which also implements robustness to noise strategies. Our numerical experiments show that Py-BOBYQA is comparable to or better than existing general DFO solvers for noisy problems. In our comparisons, we introduce a new adaptive measure of accuracy for the data profiles of noisy functions that strikes a balance between measuring the true and the noisy objective improvement.

Abstract: Display Omitted A new hybrid optimizer is proposed in which an innovative local optimal particles search strategy, which on basis of particular analysis on disadvantage of global optimal particle method, is integrated into multi-objective particle swarm optimization.Select some non-dominated solutions lied in less-crowded region of external archive and make full use of optimal particles method to guide these solutions approach the Pareto front quickly.The multi-dimensional uniform mutation operator is performed to prevent algorithm from trapping into local optimum and a dynamic archive maintenance strategy is applied to improve the diversity of solutions.For coping with the constrained conditions consist in objective functions, we adopt an efficient infeasibility degree evaluation criterion to deal with these complex problems. This paper presents a new hybrid optimizer in which an innovative optimal particles local search strategy on basis of bound optimization by quadratic approximation (BOBYQA) algorithm and exterior penalty function method is integrated into particle swarm optimization (PSO). The main goal of the approach is to improve the convergence performance of PSO, and preserve the diversity of non-dominated set. Our algorithm selects some non-dominated solutions lied in less-crowded region of external archive based upon crowding distance value to construct a leader particles set, and make full use of optimal particles method to guide leader particles approach the Pareto front quickly. Meanwhile, a local optimal particles search strategy is proposed after particular analysis on disadvantage of global optimal particle search method, and names our algorithm as LOPMOPSO. Furthermore, the multi-dimensional uniform mutation operator is performed to prevent algorithm from trapping into local optimum, and a dynamic archive maintenance strategy is applied to improve the diversity of solutions. For coping with the constrained conditions consists in objective functions, we adopt an efficient infeasibility degree evaluation criterion to deal with these complex problems. Simulation results of various kinds of benchmark functions show that our approach is highly competitive in convergence speed and generates a well distributed and accurate set of non-dominated solutions easily. The solving of a 2-D aerodynamic optimization problem further validates its speed and effectiveness.