This has happened to me, a complicated model couldn't solve the problem when a simpler one solved it in a few epochs. How is that? I believed that a more complicated model means more number of parameters and more number of parameters means a higher capability to solve a problem.
I have heard that people say, A simpler Model can perform well because it has a low variance. But, what does that even mean?