0
$\begingroup$

I'm Testing a Machine Learning model with validation data returns that return 100% correct answers, is it overfitting or the model works extremely well, do I need to continue training on more data?

I'm not sure how to interpret the result, any guidance please?

$\endgroup$

1 Answer 1

2
$\begingroup$

I'm assuming this is a classification task as you mentioned accuracy:

100% on the validation set is usually a red-flag. Check there is no data leakage in your code (inspect it at each line). Make sure your validation set has a mixture of both classes (for example: is this a very imbalanced set and your validation set has all negative class?).

There is really no way to tell if or what the problem is without some description of the data or presence of code - but 100% accuracy is usually a big indication to dig into the details to find out if there's a bias you are adding to the system.

$\endgroup$
3
  • $\begingroup$ Thank you @Oliver Foster for your remarks, I share the same exact thoughts as you, but this was one of the questions that I have found during the preparation for GCP Data Engineer old exams, and there was no data given, I know the answer to the quiz question but i thought there might be another thinking process that make me confident pointing out the right option. $\endgroup$
    – MXK
    Oct 29, 2020 at 18:34
  • $\begingroup$ With the assumption that there is no issues in the code or data pipeline (or else this information would have been provided) then 100% correct answers on the validation set is the best possible outcome. This is not overfitting as overfitting causes models to generalize poorly - which isn't the case since the validation score is perfect. $\endgroup$ Oct 29, 2020 at 18:36
  • $\begingroup$ I have assumed the same thing, but then when I answered "100% correct answers on the validation set is the best possible outcome", in the correction they have said "This is not correct because the 100% accuracy is an indicator of an overfit model. It may mean your validation data has gotten mixed in with your training data." Anyway, I was trying to learn the right thinking process about these kind of situations. I'm going to accept your answer, and thank you. $\endgroup$
    – MXK
    Oct 29, 2020 at 19:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.