1
$\begingroup$

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-all. Is this there a flaw in this thinking?

$\endgroup$
3
  • $\begingroup$ it is overfitting, best would be to use k-fold cross-validation to test how much it overfits and decide $\endgroup$
    – Nikos M.
    Jan 5, 2021 at 14:43
  • $\begingroup$ What is the Baseline i.e. if humans can simply guess with a 20% error, so that would not be a great model? You must not simply accept it i.e. do detail causal of overfitting. If Train/Test is split on Time, then this might become 25% with new data $\endgroup$
    – 10xAI
    Jan 5, 2021 at 15:52
  • 1
    $\begingroup$ Why is it so clearly overfitting? The expectation should be for the training data performance to be superior to test data performance. $\endgroup$
    – Dave
    Jun 11, 2021 at 5:02

2 Answers 2

0
$\begingroup$

Yes, assuming you haven't overfitted on the test set (which may happen after extensive hyperparameter optimization), you can assume that your model has a MAPE of 15%.

However, if you limit the overfitting, the test performance would probably go down!

$\endgroup$
0
$\begingroup$

Whether or not you can use this model depends on the specific context and requirements of your project. If a 15% error rate is acceptable for your purposes, then you might be able to use this model. However, the discrepancy between the training and testing error rates does suggest that the model may be overfitting to the training data, which means it might not generalize well to new, unseen data.

Overfitting is a problem because it means that the model is not just learning the underlying patterns in the data, but also the noise. This can lead to poor performance when the model is applied to new data.

If you decide to use this model, it would be a good idea to keep monitoring its performance over time to ensure that the error rate doesn't increase. You might also want to try different techniques to reduce overfitting, such as regularization, using a simpler model, or collecting more data.

In the end, while a 15% error rate might be acceptable depending on the context, the discrepancy between the training and testing error rates is a cause for concern and suggests that the model might not perform well on new data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.