Search results
Results from the WOW.Com Content Network
In machine learning, hyperparameter optimization [1] or tuning is the problem of choosing a set of optimal hyperparameters for a learning algorithm. A hyperparameter is a parameter whose value is used to control the learning process, which must be configured before the process starts.
In machine learning, a hyperparameter is a parameter that can be set in order to define any configurable part of a model's learning process. Hyperparameters can be classified as either model hyperparameters (such as the topology and size of a neural network) or algorithm hyperparameters (such as the learning rate and the batch size of an optimizer).
A training data set is a data set of examples used during the learning process and is used to fit the parameters (e.g., weights) of, for example, a classifier. [9] [10]For classification tasks, a supervised learning algorithm looks at the training data set to determine, or learn, the optimal combinations of variables that will generate a good predictive model. [11]
In the second part, test functions with their respective Pareto fronts for multi-objective optimization problems (MOP) are given. The artificial landscapes presented herein for single-objective optimization problems are taken from Bäck, [1] Haupt et al. [2] and from Rody Oldenhuis software. [3]
English: In hyperparameter optimization with random search, the model is trained with randomly chosen hyperparameter values. The performance in relation to hyperparameters (colored lines, better performance = blue) does not influence the choice of trials. Finally, the model with the best performance is selected. In this example, 100 trials were ...
English: For both hyperparameters of a model, a discrete set of values to search is defined (here, 10 values). In hyperparameter optimization with grid search, the model is trained using each combination of hyperparameter values (100 trials in this example) and the model performance (colored lines, better performance = blue) is saved.
After these steps, practitioners must then perform algorithm selection and hyperparameter optimization to maximize the predictive performance of their model. If deep learning is used, the architecture of the neural network must also be chosen manually by the machine learning expert.
Once an algorithm is chosen, optimizing it through hyperparameter tuning is essential to enhance efficiency and accuracy. [9] Techniques such as grid search or Bayesian optimization are employed, and engineers often utilize parallelization to expedite training processes, particularly for large models and datasets. [10]