Open AccessDissertation
High-Dimensional Bayesian Multi-Objective Optimization
David Gaudrie
- 28 Oct 2019
7
TL;DR: To improve the attainment of Pareto optimal solutions, an infill criterion is tailored to direct the search towards a user-desired region of the objective space or, in its absence, towards the Pare to front center introduced in this work.
read more
Abstract: This thesis focuses on the simultaneous optimization of expensive-to-evaluate functions that depend on a high number of parameters. This situation is frequently encountered in fields such as design engineering through numerical simulation. Bayesian optimization relying on surrogate models (Gaussian Processes) is particularly adapted to this context.The first part of this thesis is devoted to the development of new surrogate-assisted multi-objective optimization methods. To improve the attainment of Pareto optimal solutions, an infill criterion is tailored to direct the search towards a user-desired region of the objective space or, in its absence, towards the Pareto front center introduced in our work. Besides targeting a well-chosen part of the Pareto front, the method also considers the optimization budget in order to provide an as wide as possible range of optimal solutions in the limit of the available resources.Next, inspired by shape optimization problems, an optimization method with dimension reduction is proposed to tackle the curse of dimensionality. The approach hinges on the construction of hierarchized problem-related auxiliary variables that can describe all candidates globally, through a principal component analysis of potential solutions. Few of these variables suffice to approach any solution, and the most influential ones are selected and prioritized inside an additive Gaussian Process. This variable categorization is then further exploited in the Bayesian optimization algorithm which operates in reduced dimension.
read more
Chat with Paper
AI Agents for this Paper
Find similar papers on Google Scholar, PubMed and Arxiv
Write a critical review of this paper
Analyze citations of this paper to find unaddressed research gaps
Citations
•Posted Content
The Parallel Knowledge Gradient Method for Batch Bayesian Optimization
Jian Wu,Peter I. Frazier +1 more
TL;DR: It is demonstrated that the parallel knowledge gradient method finds global optima significantly faster than previous batch Bayesian optimization algorithms on both synthetic test functions and when tuning hyperparameters of practical machine learning algorithms, especially when function evaluations are noisy.
43
Revisiting Bayesian Optimization in the light of the COCO benchmark
Rodolphe Le Riche,Victor Picheny +1 more
TL;DR: In this article, a large investigation about the effects on the performance of (Gaussian process based) BO of common and less common design choices is carried out with the established COCO (COmparing Continuous Optimizers) software, and it is found that a small initial budget, a quadratic trend, high-quality optimization of the acquisition criterion bring consistent progress.
23
•Posted Content
Multi-Objective Bayesian Optimization over High-Dimensional Search Spaces.
TL;DR: MORBO as mentioned in this paper performs local Bayesian optimization within multiple trust regions simultaneously, allowing it to explore and identify diverse solutions even when the objective functions are difficult to model globally, which significantly advances the state-of-the-art in sample efficiency for several high-dimensional synthetic and real-world multi-objective problems, including a vehicle design problem with 222 parameters.
2
Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey
TL;DR: In this article , a comprehensive survey of studies of design automation techniques for deep learning models targeting edge computing is presented, which offers an overview and comparison of key metrics that are used commonly to quantify the proficiency of models in terms of effectiveness, lightness and computational costs.
2
References
•Book
Genetic algorithms in search, optimization, and machine learning
David E. Goldberg
- 01 Sep 1988
TL;DR: In this article, the authors present the computer techniques, mathematical tools, and research results that will enable both students and practitioners to apply genetic algorithms to problems in many fields, including computer programming and mathematics.
•Book
The Nature of Statistical Learning Theory
Vladimir Vapnik
- 01 Jan 1995
TL;DR: Setting of the learning problem consistency of learning processes bounds on the rate of convergence ofLearning processes controlling the generalization ability of learning process constructing learning algorithms what is important in learning theory?
46K
Regression Shrinkage and Selection via the Lasso
TL;DR: A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.
A fast and elitist multiobjective genetic algorithm: NSGA-II
TL;DR: This paper suggests a non-dominated sorting-based MOEA, called NSGA-II (Non-dominated Sorting Genetic Algorithm II), which alleviates all of the above three difficulties, and modify the definition of dominance in order to solve constrained multi-objective problems efficiently.
•Proceedings Article
Auto-Encoding Variational Bayes
Diederik P. Kingma,Max Welling +1 more
- 01 Jan 2014
TL;DR: A stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case is introduced.