Benchmarking Derivative-Free Optimization Algorithms

Question

1. What are the contributions mentioned in the paper "Benchmarking derivative-free optimization algorithms" ?

2. What have the authors stated for future works in "Benchmarking derivative-free optimization algorithms" ?

3. How many simplex gradients can solve the problem?

4. What is the second class of problems?

Accepted Answer

The authors propose data profiles as a tool for analyzing the performance of derivativefree optimization solvers when there are constraints on the computational budget.. The authors use performance and data profiles, together with a convergence test that measures the decrease in function value, to analyze the performance of three solvers on sets of smooth, noisy, and piecewise-smooth problems.

Accepted Answer

Below the authors outline four other possible future research directions.. The authors plan to validate this claim in future work.. Results for additional solvers can be added easily.. Their computational experiments used default input and algorithmic parameters, but the authors are aware that performance can change for other choices.

Accepted Answer

Since the problems in their benchmark sets have at most 12 variables, the authors set µf = 1300 so that all solvers can use at least 100 simplex gradients.

Accepted Answer

The second class of problems mimics simulations that are defined by an iterative process, for example, solving to a specified accuracy a differential equation where the differential equation or the data depends on several parameters.

Accepted Answer

Users with expensive function evaluations are often interested in a convergence test that measures the decrease in function value.

Accepted Answer

The authors present the data profiles for τ = 10−k with k ∈ {1, 3, 5, 7} because the authors are interested in the short-term behavior of the algorithms as the accuracy level changes.

Accepted Answer

Their interest in derivative-free methods is motivated in large part by the computationally expensive optimization problems that arise in DOE’s SciDAC initiative.

Accepted Answer

Arguments for a non-stochastic noise function were presented in Section 4, but here the authors add that a significant advantage of using a non-stochastic noise function in benchmarking is that this guarantees that the computational results are reproducible up to the precision of the computations.

Accepted Answer

These expensive optimization problems arise in science and engineering because evaluation of the function f often requires a complex deterministic simulation based on solving the equations (for example, nonlinear eigenvalue problems, ordinary or partial differential equations) that describe the underlying physical phenomena.

Accepted Answer

As usual, there is a limit µf on the total number of function evaluations, and tp,s = ∞ if the convergence test (2.2) is not satisfied after µf evaluations.

Accepted Answer

The convergence test (2.2) can also be written asf(x0)− f(x) ≥ (1− τ)(f(x0)− fL),and this shows that (2.2) requires that the reduction f(x0)− f(x) achieved by x be at least 1− τ times the best possible reduction f(x0)− fL.The convergence test (2.2) was used by Elster and Neumaier [6] but with fL set to an accurate estimate of f at a global minimizer.

Accepted Answer

The authors can obtain this information by letting tp,s be the number of function evaluations required to satisfy (2.2) for a given tolerance τ , since thends(α) = 1 |P|size { p ∈ P : tp,s ≤ α } is the percentage of problems that can be solved with α function evaluations.

Accepted Answer

All termination tolerances were set as described in Section 3 so that solvers effectively terminate only when the limit µf on the number of function evaluations is exceeded.

Benchmarking Derivative-Free Optimization Algorithms

Chat with Paper

AI Agents for this Paper

Most frequently asked questions

1. What are the contributions mentioned in the paper "Benchmarking derivative-free optimization algorithms" ?

2. What have the authors stated for future works in "Benchmarking derivative-free optimization algorithms" ?

3. How many simplex gradients can solve the problem?

4. What is the second class of problems?

5. What is the common question asked by users with expensive function evaluations?

6. Why do the authors present the data profiles for = 10k with k?

7. What is the motivation for the interest in derivative-free methods?

8. What is the advantage of using a non-stochastic noise function in benchmarking?

9. What is the cost of evaluating a derivative-free optimization solver?

10. What is the limit on the number of function evaluations?

11. What is the convergence test for f?

12. How many function evaluations can be used to solve a given problem?

13. What are the termination tolerances for the solvers?

Figures

Citations

Derivative-free optimization: a review of algorithms and comparison of software implementations

Nuclear Energy Density Optimization

Simulation optimization: A review of algorithms and applications

COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting

Performance indicators in multiobjective optimization

References

Benchmarking optimization software with performance profiles

Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods ∗

Analysis of Generalized Pattern Searches

A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems

CUTEr and SifDec: A constrained and unconstrained testing environment, revisited

Related Papers (5)

Introduction to derivative-free optimization

Benchmarking optimization software with performance profiles

Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods ∗

A simplex method for function minimization

On the Convergence of Pattern Search Algorithms