I am working with a large data set (~9M rows with 20+ features). Is it ok to tune via grid search on a fraction of the data (~100k rows) to determine optimal hyperparameters? This is mostly for choosing max_features, min_samples, max_depth. Trees and learning rate come later. Will I get different results tuning the fraction versus the whole data set?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
You should never train or do grid search on your entire data set, since it will lead to overfitting and reduce the accuracy of your model on new data. What you have described is actually the ideal approach: do grid search / training on a subset of your data. Yes, your model will get different results vs if you used the entire set of data, but your model will be much stronger because of it.
For more details on why you would want to split up / sample your data, see this quesiton: https://stats.stackexchange.com/questions/19048/what-is-the-difference-between-test-set-and-validation-set
answered Apr 21, 2016 at 13:11
-
$\begingroup$ I understand this concept. So I have a random 70% of my data in Train. I cant run grid search on the entire 70% because its too big. Ideally I would like to run grid search on 5%, will I get different tuning results on the 5% versus running grid search on the entire 70%? $\endgroup$ Apr 21, 2016 at 15:17
-
$\begingroup$ Yes, the optimal tuning parameters could change somewhat, but that shouldn't matter. What matters is that you have a strong model. 5% of 9M is a sufficient record size to build a strong model. $\endgroup$ Apr 21, 2016 at 15:26