For a small dataset that has less than 100 samples, I have run a model A. The result of R2 square for both test and train set is about 82%. But when I perform k fold cross validation on X, y, the result is negative! What is the cause?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Let's adress the negative R2 score first, this can happen when the model is predicting worse than a dummy predictor which is the horizontal straight line that represents the mean of your target.
With less than 100 samples, the dataset may not have enough variability to provide accurate scores of the model. Cross-validation is usually done to have more robust estimate of model performance but here since your dataset is small, cross validation might be a problem, because this dataset will be split in folds (even smaller) and scores can randomly differ based on the split,
In depth EDA will help gaining relevant insights in the dataset to select the best strategy (regarding data preprocessing, data augmentation and model selection)
