It's all about the bias-variance trade-off (https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff). Simple models tend to have a high bias, but low variance and complex models tend to have the opposite effect. It's about searching for the right balance. I suggest you go and buy 'Elements of statistical learning', which is a superb reference for people who want to dive in ML.
Together with the no free lunch theorem, it's hard to determine whether your model is good or bad, except for trial and error. I usually use 2 metrics: the Weighted Mean Absolute Error (https://forums.fast.ai/t/how-to-calculate-weighted-mean-absolute-error-wmae/8575) for the bias component and a simple metric where I divide the MAPE(mean absolute percentage error) into buckets (how many predictions were close to 0% mape and how many were far worse). It gives you an idea of how the variance of a model is. A good visual representation of bias and variance is here:http://scott.fortmann-roe.com/docs/BiasVariance.html.
Another thing I use, is the Zero Rule algorithm to compare it to this one.
There is so much more to explain, so I would suggest to keep digging into articles, papers, sites like this,...!