3
$\begingroup$

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect the performance of my model? If yes, how do they relate?

$\endgroup$
1
$\begingroup$

You need to look for the differences in the training loss and the cross-validation & test losses. If those are low, it means the model performs fairly well. Ideally, train loss should be roughly equal to the cross-validation and test losses. If not, the model is overfitted.

This difference also hints at an insignificant overlap between the train data points and the cross-validation and test data points. Such features are said to be unstable. The model in such a case only gets to see the data points in the train data and not those in the cross-validation and test data, and is simply overfitted thereby. It, hence, performs poorly. You can verify the same by computing the percentage of data points present in the cross-validation and test data out of those in the train data for different features in your dataset.

$\endgroup$
  • $\begingroup$ It's not that simple; a large difference between train & test loss may be a sign of underfitting, too. $\endgroup$ – desertnaut Oct 13 '20 at 15:51
  • 2
    $\begingroup$ @desertnaut Yeah, you got that right. But it ain't that difficult as well! $\endgroup$ – Shayan Shafiq Oct 14 '20 at 4:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy