0
$\begingroup$

So I understand that overfitting is when you have for example a good accuracy for the training dataset and bad one for the testing dataset, but why would I even check the accuracy for the training dataset? If I have a good accuracy on the testing dataset that means I'm most likely not overfitting, right? (Assuming that we make sure that the model doesn't train on any testing data)

I have another question: Can oversampling using SMOTE cause overfitting (good accuracy on the testing dataset but in reality it is overfitting?) SMOTE draws a line and makes new points on it so it doesn't duplicate the data.

$\endgroup$

1 Answer 1

2
$\begingroup$
  • First point: in general it's risky to use accuracy in order to measure performance, especially if there is class imbalance. F1-score would be a better option in general.
  • Significantly lower performance on the test set compared to the training set is indeed the main indication of overfitting. It's worth keeping in mind that Overfitting means that the model captures too many details which happen by chance in the training set, a problem more likely to happen if the training set is too small and/or the model is too complex.

but why would I even check the accuracy for the training dataset?

Simply to detect overfitting, because it's a very common issue.

If I have a good accuracy on the testing dataset that means I'm most likely not overfitting, right? (Assuming that we make sure that the model doesn't train on any testing data)

This is correct, but very often we don't know how to define "good": for a very hard task, a F1-score of 0.5 could be very good for instance, but for some very easy task an accuracy of 0.99 might be bad. In other words, unless there is some benchmark performance to compare against, there's no easy way to know if the performance is good.

I have another question: Can oversampling using SMOTE cause overfitting (good accuracy on the testing dataset but in reality it is overfitting?) SMOTE draws a line and makes new points on it so it doesn't duplicate the data.

Yes, totally: the points artificially created by SMOTE are not real data points, so there is a high chance that they do not have the same variations that real data points would have (whether it's noise or significant patterns). But the number of points can mislead the model, which is likely to assume that there's an important regular pattern, even though it's actually only caused by very few (possibly random) points artificially inflated by SMOTE.

$\endgroup$
1
  • $\begingroup$ Thank you so much, you answered my other question too 10 months ago or something like that <3 $\endgroup$
    – Maxi
    Dec 30, 2022 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.