# Contradictory learning curves in cross validation

I'm fitting gradient boosted decision trees (lightgbm) to model a regression problem. The data is extremely noisy, $$R^2 \approx 0$$.

I'm trying to improve the fitting procedure using 10 fold cross validation.

I plot the learning curves, that is, training score and validation score as a function of number of iterations (=number of trees) for each train/validation split.

The train scores are increasing in each split.

But for half the splits, the test scores are increasing, and for the other half they're decreasing.

Does this mean that on half the splits I'm underfitting, the other half I'm overfitting? If so, what can I do?

• You can post your curve by taking a screenshot and using the image widget so we can undertand better Jul 26 at 13:11
• Will post when I get access, but imagine on a good split train_score(n) = sqrt(n), test_score(n) = 0.5 sqrt(n). On a bad split train_score(n) = sqrt(n), test_score(n) = -sqrt(n) Jul 26 at 13:48