# A way to Identify tuning parameters and their possible range

I am a novice in Machine Learning. But when I started learning, I figure out that all the methods have some tuning parameters and those parameters take a range of possible values. By grid searching, we identify a set of these parameters that optimize some function. But is there any way to find the possible domain of the tuning parameters? This would definitely save my time and the computer's job. In addition, some methods such as xgboost have loads of tuning parameters. Is there any way to know which one to tune and which one to leave as it is. I have been using sklearn python library.

• This is where domain knowledge comes in, if you know something about your data beforehand you can use this to decrease model selection time. – user2974951 Sep 25 '18 at 13:23

I always first try to see how the default parameters perform. Then from the documentation or some reading, you can see what is each parameter global influence (by influence I mean maybe increasing parameter X means complexifying the model, or parameter Y means increasing the convergence speed towards a solution). Depending on the first result you get, pick up one parameter, the one that seem to have the most influence on the model, and make it vary a bit in the way that make sense from your first results. If things improve on the validation set, keep moving the value this way, if not do the opposite. Often times you get good results without tuning every single parameter.