Optimally weighted ensembles of surrogate models for sequential parameter optimization

Leiden Repository

Optimally weighted ensembles of surrogate models for sequential parameter optimization

Type: Doctoral Thesis
Title: Optimally weighted ensembles of surrogate models for sequential parameter optimization
Author: Echtenbruck, M.M.
Issue Date: 2020-07-02
Keywords: Machine Learning
Surrogate Modelling
Ensemble Learning
Sequential Optimization
Ensemble-Based Learning
Black-box Global Optimization
Abstract: It is a common technique in global optimization with expensive black-box functions to learn a surrogate-model of the response function from past evaluations and use it to decide on the location of future evaluations.In surrogate-model-assisted optimization, selecting the right modeling technique without preliminary knowledge about the objective function can be challenging. It might be beneficial if the algorithm trains many different surrogate models and selects the model with the smallest training error. This approach is known as model selection.In this thesis, a generalization of this approach is developed. Instead of choosing a single model, the optimal convex combinations of model predictions is used to combine surrogate models into one more accurate ensemble surrogate model.This approach is studied in a fundamental way, by first evaluating minimalistic ensembles of only two surrogate models in detail and then proceeding to ensembles with more surrogate models.Finally, the approach is adopted and evaluated in the context of sequential parameter optimization. Besides discussing the general strategy, the optimal frequency of learning the convex combination of weights is investigated.The results provide insights into the performance, scalability, and robustness of the approach.
Promotor: Supervisor: Bäck T.W.H., Bartz-Beielstein T. Co-Supervisor: Emmerich M.T.M.
Faculty: Faculty of Science
University: Leiden University
Handle: http://hdl.handle.net/1887/123184
 

Files in this item

Description Size View
application/pdf Full Text 4.157Mb Under embargo until 2022-01-02
application/pdf Cover 6.976Mb View/Open
application/pdf Title pages_Contents 246.0Kb View/Open
application/pdf Chapter 1 706.3Kb View/Open
application/pdf Chapter 2 1.925Mb View/Open
application/pdf Chapter 3 537.9Kb View/Open
application/pdf Chapter 4 1.486Mb View/Open Full text at publisher site
application/pdf Chapter 5 635.1Kb Under embargo until 2022-01-02 Full text at publisher site
application/pdf Chapter 6 282.9Kb View/Open Full text at publisher site
application/pdf Appendix_Bibliography_Glossary 880.0Kb View/Open
application/pdf Summary in Dutch 195.9Kb View/Open
application/pdf Curriculum Vitae 153.0Kb View/Open
application/pdf Propositions 131.6Kb View/Open

This item appears in the following Collection(s)