I'm fitting gradient boosted decision trees (lightgbm) to model a regression problem. The data is extremely noisy, $R^2 \approx 0$.

I'm trying to improve the fitting procedure using 10 fold cross validation.

I plot the learning curves, that is, training score and validation score as a function of number of iterations (=number of trees) for each train/validation split.

The train scores are increasing in each split.

But for half the splits, the test scores are increasing, and for the other half they're decreasing.

Does this mean that on half the splits I'm underfitting, the other half I'm overfitting? If so, what can I do?

  • $\begingroup$ You can post your curve by taking a screenshot and using the image widget so we can undertand better $\endgroup$
    – BeamsAdept
    Jul 26 at 13:11
  • 1
    $\begingroup$ Will post when I get access, but imagine on a good split train_score(n) = sqrt(n), test_score(n) = 0.5 sqrt(n). On a bad split train_score(n) = sqrt(n), test_score(n) = -sqrt(n) $\endgroup$
    – user357269
    Jul 26 at 13:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.