# Bias and variance in the model o in the predictions?

This topic confuses me. In the literature or articles, when talking about bias and variance in automatic learning, specifically in cross-validation, do they refer to the high bias (underfitting) and high variance (overfitting) in the model? Or do they refer to the bias and variance of the predictions obtained in the iterations of the cross-validation? How to handle each case?

• I think this question is related to the „bias variance trade-off“ en.m.wikipedia.org/wiki/Bias–variance_tradeoff – Peter Jul 25 '19 at 22:04

## 2 Answers

In some cases, you may have a model that's a black box - you feed input features and get output predictions, without knowing or caring what happens in the middle. In those situations, the model is, in a way, defined by its output - two models that produce the same predictions are indistinguishable from one another, even though they may be entirely distinct models. In these situations, saying a model is biased or has high variance is equivalent to saying the predictions of the model are biased or have high variance. It's acceptable to describe both the model and its output in terms of bias/variance. Cross validation provides an unbiased estimate of bias/variance for your model/predictions.

• First.Thanks for your answer. – SRG Jun 25 '19 at 20:14
• Other question, in this book: books.google.com.mx/…, on page 13, the following is mentioned: "To reduce the influence of randomness introduced by data split, the k-fold cross validation can be repeated t times..." I understand that this randomness produces high variance in the results of the predictions, then, that high variance produces an model with overfitting? – SRG Jun 25 '19 at 20:14

I finally understood "Bias and Variance" with Logistic Regression (LR). LR is known to have a high bias but low variance. For example, LR accuracy might be only at 90% while you'd get 95% with a Decision Tree (tends to overfit). But with LR in production feeding new data, you're very likely to see 90% accuracy ... while the Decision Tree had massive accuracy swings (variance) with new data (70% - 95%).

Hope that helps.