Every time AI gurus talk about fine tuning hyper parameters, they more or less say it's trial and error. But can't we make an AI to tell AI what its hyper parameters should be?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Yes, you can make an AI for that. By AI, you mean algorithm which finds hyperparameters efficiently. There are many ways to find hyperparameters which would then come under the AI like using bandits to find hyperparameters, Bayesian methods to find hyperparameters and many other methods exist. Search for hyperparameter optimization and you will find a ton of methods which improve upon grid search and random search.
AI in the purest sense of it, if I take, then, I will assume that you are talking about reinforcement learning since that deals with taking actions and changing its behaviour (in this case, hyperparameters). You can create an AI for that.
