4
$\begingroup$

What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.

$\endgroup$

closed as too broad by Stephen Rauch, Nain, Aditya, Siong Thye Goh, Toros91 Apr 30 '18 at 7:42

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

3
$\begingroup$

You want to decide this based on how well your model performs and generalizes. If your model is underfitting, you want to increase your model's complexity, increasing variance and decreasing bias. If your model is overfitting, you want to regularize the model and/or feed it more training data, decreasing variance and increasing bias.

$\endgroup$
3
$\begingroup$

In my opinion you should do the following:

1) Try to minimize the bias as much as possible. This can usually be achieved by choosing a more complex model. This step is done to ensure that your model has enough capacity to solve the problem. This will, however, cause your model to overfit.

2) Regularize the above model to reduce its variance. This can be done through norm penalties (L1, L2, etc.), early stopping, or other model specific techniques (dropout for neural networks, pruning for decision trees, etc.).

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.