Are there any advanced packages that allow automated hyperparameter tuning for neural networks and traditional machine learning algorithms like XGBoost, random forest (using method like Bayesian, random search etc. that could allow for faster discovery of the optimal parameters)? I have heard of hyperopt, but it seems there are some problems, and I am not sure it can train traditional machine learning algorithms?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ New packages come out all the time; here are two: FAR-HO, BayesianOptimization. I would recommend using random search when applicable; it is simple and performant. $\endgroup$– EmreJan 26, 2018 at 6:49
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
There are a number of methods to automate the optimization of your hyper-parameters, such as GridSearch and RandomSearch which the article you linked discusses briefly.
The main reason to choose one over the other is if you want the best possible parameters, and do not care how long it takes to get them: go for GridSearch. On the other hand, if you do not want the optimization to take a long time, but still want some good parameters then go for RandomSearch.
These two implementations in scikit-learn are not exactly "advanced packages", but they will get the job done for any model in sckit-learn (Random Forest, MLPClassifiers, etc). Emre's comment also has some pretty cool advanced packages which are scikit-learn or TensorFlow compatible.
answered Jan 26, 2018 at 11:48
$\begingroup$
$\endgroup$
If you want to go for Bayesian optimization, OPTUNA would be your best bet. It has several advantages over HyperOpt.
