This has happened to me, a complicated model couldn't solve the problem when a simpler one solved it in a few epochs. How is that? I believed that a more complicated model means more number of parameters and more number of parameters means a higher capability to solve a problem.

I have heard that people say, A simpler Model can perform well because it has a low variance. But, what does that even mean?

  • $\begingroup$ What do you call "a simpler model" ? A model using less variables ? A model easier to implement ? With less parameters ? $\endgroup$ – BeamsAdept Oct 7 '20 at 13:02

High variance means that your model's performance varies from data to data, which is bad for the model.

For example:
You used a polynomial classifier with high degree, so it will overfit your training data resulting in good accuracy, but when tried on a different dataset it will yield less accuracy , resulting in variance.

This will help you understand

  • The image on the right is way too complex which results in overfitting.

  • The middle image shows a simpler model than the image on right, but fits the data better ie. generalize the data better


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.