Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges

Hyperparameter optimization: Foundations, algorithms, best practices, and open challenges Abstract Most machine learning algorithms are configured by a set of hyperparameters whose values must be carefully chosen and which often considerably impact performance. To avoid a time‐consuming and irreproducible manual process of trial‐and‐error to find well‐performing hyperparameter configurations, various automatic hyperparameter optimization (HPO) methods—for example, based on resampling error esti...