# Training with a subset of data: relationship between subset size and training metric?

I have a not-quite linear regression problem which I am investigating. The data set is fairly large, with ~6000 samples and ~2100 features.

By performing 5-fold cross-validation on different sized subsets of the data set, I have found a strong relationship between the fraction of the data set used and the value of both the R-squared metric and the RMSE metric.

EDIT for clarity: for a fraction of 0.01, I am taking 1/100 of the samples (~60) and then performing repeated 5-fold cross-validation as if this were the whole data set. The values shown are means.

It appears that RMSE varies linearly with the reciprocal of the fraction used (1/Frac), and the R-squared regresses very well against a second-order polynomial of 1/Frac.

The raw results are as follows:

Frac   1/Frac  Test R2      Test RMSE
0.01   100     -1.65628     1.85292
0.013  75      -0.71208     1.71476
0.02   50      -0.00786883  1.38874
0.05   20       0.535839    1.06872
0.10   10       0.626532    1.00598
0.20   5        0.702421    0.95058
0.30   3.33     0.745277    0.860082
0.50   2        0.772211    0.86548


Relationships:

My questions are:

1. Is this a well-known relationship?
2. Does it have a name (so I can research further)?
3. Is there a theoretical basis for it?

Do I understand correctly from your test RMSE, that the error is lower as you increase the size of your fraction used in training?

If you do 5-fold cross validation using a fraction of 0.01, that implies here that you are only a total fraction of only 0.05 of your data? So 0.04 for training and 0.01 for testing.

It should not come as a surprise then, that the performance improves with increased amounts of data used for training. Your RMSE values increase with fraction size - I would normally expect such a correlation (assuming your data is fairly coherent).