On the selection of surrogate models in evolutionary optimization algorithms

TL;DR: This work compares four meta-modeling techniques, polynomial approximation, kriging, radial basis functions and support vector regression, in different aspects such as accuracy, robustness, efficiency, and scalability with the aim of selecting the most suitable one to be combined with evolutionary optimization algorithms.

Abstract: Since many real-world problems are related to the satisfaction of at least one goal, several optimization techniques have been proposed in the past. However, traditional optimization techniques are computationally expensive and are normally highly susceptible to some characteristics such as high dimensionality, non-differentiability, non-linearity, highly expensive function calculation, among others. Evolutionary algorithms are bio-inspired meta-heuristics that have shown flexibility, adaptability and good performance when solving these sort of problems. In order to achieve acceptable results, some problems usually require several evaluations of the optimization function. However, when each of these evaluations represents a high computational cost, these problems remain intractable even by these meta-heuristics. To reduce the computational cost in expensive optimization problems, some researchers have replaced the real optimization function with a computationally inexpensive surrogate model. Despite there are comparison studies among these techniques, these studies focused on revised the accuracy of the meta-model for the problem at hand, but neither its suitability to be used with evolutionary algorithms, nor its scalability in the variable design space. In this work, we compare four meta-modeling techniques, polynomial approximation, kriging, radial basis functions and support vector regression, in different aspects such as accuracy, robustness, efficiency, and scalability with the aim to identify advantages and disadvantages of each meta-modeling technique in order to select the most suitable one to be combined with evolutionary optimization algorithms.