Ημερομηνία : 01/07/2021
Συγγραφείς : Lytra G., Mavrogiorgou A., Kiourtis A., & Kyriazis D.
IOSR Journal of Computer Engineering (IOSR-JCE), Volume 5, Issue 4, pp. 34-50, IOSR.
Background: Machine learning has taken the technological world by storm in recent years. Every expert that needs to cope with a specific problem, demands to develop a model that will be able to cover all the problem’s needs in combination with all the available resources that come by. Along the way of this process, a lot of challenges may come up. One of these challenges refers to the selection of the most appropriate hyperparameters that should be used for the construction of the most efficient model. This process, called hyperparameter optimization, is considered to be extremely vital. Even though the created models are proved to be effective in both performance and execution time, the same models can be rendered rather than useless without the appropriate hyperparameters’ selection. It is a fact that hyperparameter optimization can really help a model to shine and exploit its capabilities to the fullest. However, since every problem has its uniqueness and complexity, domain knowledge is necessary for choosing the appropriate hyperparameters in each different case. Hence, the need of implementing methodologies that automatically solve this issue is on the rise. Materials and Methods: This study tries to fill in this gap, by following an experimental procedure to extract information regarding the appropriate hyperparameters on various supervised (classification and regression) learning models. Various datasets with diverse features and characteristics are exploited, which could assist the successful automation of machine learning processes, whereas already existing optimization frameworks are fully utilized.
Results: The conducted study resulted into the extraction of both reliable and generalized results that cover a variety of diverse machine learning problems deriving from various sectors.
Conclusion: The study led to doubts on hyperparameter optimization as a practice that should occur in all cases of development of machine learning models. There are some factors that appear to affect the whole process, whereas the same set of factors is what should help the user decide whether performing hyperparameter optimization worths its trade-offs or not.