2
$\begingroup$

I have a dataset of 300 000 rows and an ensemble model, which include grid search to find the best params of every algorithm. Unfortunately the grid search needs to long and I have problems to implement gpu running for the different algorithms (xgb,lightgbm..). Also the models which are from scikit-learn like random-forest dont run on gpu. The idea is now, instead of 300 000 rows, I will create a small dataset with max. 500 rows, which need less time than the full dataset.

  1. Could the minimal required sample size calculation help?

  2. How big should the sample be? to get a good data distribution of the big dataset?

Improve this question
$\endgroup$
2
$\begingroup$

This idea of using a small sample to the data set to search for the hyperparameters is called multi-fidelity methods. A good starting point is the book by Frank Hutter, Lars Kotthoff, Joaquin Vanschoren Automated machine learning: methods, systems, challenges which is open access.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you very much I will read about that. Never heard that before. $\endgroup$ – martin Sep 1 '20 at 20:38
1
$\begingroup$

You could also use Sklearn Randomized Search, which is a bit less efficient than Grid Search, since it won't test all conbinaisons, but could find one of the best combinaisons, and then you manually end your tuning by tweaking slightly parameters based on what Randomized Search gave you. I think it would be more profitable than downsampling your dataset to make the GridSearch work, since you'll use the best Tuning method, but on a way less efficient Dataset.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you I will check it out. $\endgroup$ – martin Aug 11 '20 at 6:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.