2
$\begingroup$

When using Neural Networks for image processing I learned a rule of thumb: to avoid overfitting, supply at least 10 training examples for every neuron.

Is there a similar rule of thumb for XGBoost, presumably taking into account the number of features and estimators?

$\endgroup$
0
$\begingroup$

It's completely true that the number of examples should be related to features. But it is not only the number of features because the range of a number(max-min and count of different number) is also important. On the other hand, if you have noise you need more examples, so it's related to your dataset.

$\endgroup$
  • $\begingroup$ look like you want to do "ensemble". search it if I am right $\endgroup$ – parvij Jul 22 '18 at 16:13
0
$\begingroup$

This is not only a number of sample, this is also a question of depth.

The higher depth you have, the more you're likely to overfit.

You can reduce the overfit by adding a high number of trees, that allows you to "steady" your algorithm

$\endgroup$
  • 1
    $\begingroup$ Or by other parameters that reduce overfitting of XGBoost, for example minimum weight to create split (min_child_weight in XGBoost package in Python). $\endgroup$ – Itamar Mushkin Jan 19 at 6:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.