I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-all. Is this there a flaw in this thinking?

  • $\begingroup$ it is overfitting, best would be to use k-fold cross-validation to test how much it overfits and decide $\endgroup$
    – Nikos M.
    Jan 5 at 14:43
  • $\begingroup$ What is the Baseline i.e. if humans can simply guess with a 20% error, so that would not be a great model? You must not simply accept it i.e. do detail causal of overfitting. If Train/Test is split on Time, then this might become 25% with new data $\endgroup$
    – 10xAI
    Jan 5 at 15:52
  • 1
    $\begingroup$ Why is it so clearly overfitting? The expectation should be for the training data performance to be superior to test data performance. $\endgroup$
    – Dave
    Jun 11 at 5:02

Yes, assuming you haven't overfitted on the test set (which may happen after extensive hyperparameter optimization), you can assume that your model has a MAPE of 15%.

However, if you limit the overfitting, the test performance would probably go down!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.