I am confused about Bias and overfitting as I see it sounds similar but not able to explain differene?

  • $\begingroup$ please take a look around before asking a new question, e.g. here $\endgroup$ – oW_ May 7 at 18:03

Well, I can't express enough that there are many many many articles and questions on bias-variance, over-fitting and under-fitting.

But let me also try to explain in simple words for you.

"Bias" as a word means- favoring a thing more than another. Like we heard Amazon's alexa doesn't respond to female voices/commands. so it was biased towards men.

Now if want to make a model which is able to predict on unseen data i simply don't want it to be biased like i did't want this system to favor male's voice over female'voice. If its biased it is essentially giving me an error. Its making more assumptions about my target function(which is taking voice commands here).

Now since its assuming a lot and giving me an error, its rather under-fitting.However it actually is not only high bias but+low variance leads to under-fitting but i am keeping variance aside for now. Now, there can be many reason for high bias and low variance.I am not going into details on reasons w.r.t. training/test size.

Now, if my model is able to predict the data correctly on given test data which is a good news,it means there are no errors ,it mean there is less bias. however may be its giving me no error because it learned the "training" data so well, captured every scenario/noise. Now again comes the variance, that there is high variance here, which together with low bias shall lead to over-fitting. it means it is well able to predict on the data i gave it to train but if i feed it new data it won't be able to predict it right i.e. generalization.

Now you might wanna understand the variance also. take again the simple dictionary meaning of variance - which is "the fact or quality of being different, divergent, or inconsistent." now apply this definition in relation with target function.

Please note this is a very very vague explanation for you to follow and there are standard definitions for the concept which i suggest you to follow.

Now there are way better explanation- https://machinelearningmastery.com/gentle-introduction-to-the-bias-variance-trade-off-in-machine-learning/ https://stats.stackexchange.com/questions/4284/intuitive-explanation-of-the-bias-variance-tradeoff

Here is a good paper-


Most precise visualization is below-

enter image description here

| improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.