Highest Voted 'bias' Questions

20

votes

5answers

9k views

Why underfitting is called high bias and overfitting is called high variance?

I have been using terms like underfitting/overfitting and bias-variance tradeoff for quite some while in data science discussions and I understand that underfitting is associated with high bias and ...

variance bias

asked Feb 14 '19 at 14:33

Vaibhav Thakur

1,7932 gold badges7 silver badges9 bronze badges

8

votes

1answer

817 views

Question on bias-variance tradeoff and means of optimization

So I was wondering how does one, for example, can best optimize the model they are trying to build when confronted with issues presented by high bias or high variance. Now, of course, you can play ...

machine-learning variance bias

asked Apr 12 '18 at 20:19

Zer0k

1554 bronze badges

5

votes

2answers

775 views

What are bias and variance in machine learning?

I am studying machine learning, and I have encountered the concept of bias and variance. I am a university student and in the slides of my professor, the bias is defined as: $bias = E[error_s(h)]-...

machine-learning dataset variance bias

asked Aug 12 '20 at 8:10

J.D.

5312 gold badges4 silver badges16 bronze badges

5

votes

2answers

534 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...

random-forest xgboost boosting variance bias

asked Oct 15 '19 at 13:19

K.Hua

1535 bronze badges

4

votes

2answers

140 views

Trade off between Bias and Variance [closed]

What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.

machine-learning variance bias

asked Apr 28 '18 at 7:22

deepguy

1,2714 gold badges13 silver badges34 bronze badges

3

votes

1answer

87 views

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is ...

machine-learning variance bias ensemble bagging

asked Apr 10 '20 at 8:16

chekhovana

311 bronze badge

3

votes

1answer

463 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...

overfitting variance bias

asked Jan 17 '19 at 9:40

Deer Jona

414 bronze badges

3

votes

1answer

34 views

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?

machine-learning neural-network deep-learning perceptron bias

asked Aug 2 '18 at 9:34

Developer

7591 gold badge5 silver badges10 bronze badges

3

votes

1answer

394 views

Is normalizing the validation set of time series a kind of look ahead bias?

Here's the data normalization process of a time series in a paper about stock prediction using LSTM: Split train and test set based on time (e.g. training set: 2001-2010, test set:2011-2012). This ...

time-series lstm bias data-leakage

asked Mar 9 '19 at 21:31

TQA

4912 silver badges12 bronze badges

3

votes

1answer

600 views

Correcting log-bias in the output of an XGB

I have previously worked with GAMs, where I was trying to do regression on a log-transformed variable. The log-transformation introduced a negative bias in the average of the predicted variable, and I ...

machine-learning xgboost bias

asked Feb 19 '18 at 15:18

N08

1311 bronze badge

3

votes

1answer

287 views

Learning curve using micro F-score and macro F-score

I plotted the learning curves using micro and macro F-scores for a Multinomial Naive Bayes classifier. The first plot is made using micro F-score, and the second using macro F-score. I find it quite ...

machine-learning evaluation variance bias

asked Sep 1 '19 at 23:09

Ana

311 bronze badge

2

votes

1answer

214 views

How does C have effects on bias and variance of a Support Vector Machine?

The minimization problem for SVM can be written as- $$\overset{\text{min}}{\theta} C\sum_{i = 1}^{m}{[y^icost_1(\theta^Tx^i) + (1-y^i)cost_0(\theta^Tx^i)]} + \frac12\sum_{j = 1}^n{\theta_j}^2$$ Now, ...

machine-learning svm variance bias parameter

asked Aug 16 '20 at 6:58

truth

1307 bronze badges

2

votes

4answers

1k views

Linear machine learning algorithms “often” have high bias/low variance?

In this blog, which explains the meaning of bias and variance in machine learning, there's a line under the heading "Bias-Variance Trade-Off" which says: Parametric or linear machine learning ...

linear-regression variance bias

asked Dec 14 '18 at 18:00

falcon lover

213 bronze badges

2

votes

1answer

52 views

Which between random forest or extra tree is best in a unbalance dataset?

I have an unbalanced dataset, with 3 classes, with 60% of class 1, 38% of class 2, and 2% of class 3. I don't want to generate more examples of class 3, and I cannot get more examples of class 3. The ...

machine-learning class-imbalance variance bias

asked Jun 21 '20 at 20:08

Tlaloc-ES

3171 silver badge6 bronze badges

2

votes

1answer

45 views

Why is it okay to set the bias vector up with zeros, and not the weight matrices?

We do not initialize weight matrices with zeros because the symmetry isn’t broken during the backward pass, and subsequently in the parameter updating process. But it is safe to set the bias vector up ...

neural-network deep-learning backpropagation bias weight-initialization

asked Oct 21 '20 at 20:22

truth

1307 bronze badges

2

votes

1answer

225 views

Bias-variance tradeoff and the uncertainty principle

Bias variance tradeoff seems to behave like the uncertainty principle, is it just another name for the same principle?

variance bias

asked Apr 23 '19 at 0:56

Eugen

4452 silver badges7 bronze badges

2

votes

1answer

53 views

svm optimization problem

Suppose we have the dataset: {(3,1),(3-1),(6,1),(6,-1)} {(1,0),(0,1),(0,-1),(-1,0)} the first set represent the positive label, and de second the negative. I want manually find the support vectors, ...

svm bias

asked Dec 6 '18 at 14:54

user63851

211 bronze badge

2

votes

2answers

71 views

Predictive modeling when output affects future input

Assume I have a model which predicts the outcome of number of icecream sold in a store. The model is trained on data for the last 5 years while keep the last year as a validation set and has produced ...

machine-learning predictive-modeling bias methodology concept-drift

asked Oct 19 '20 at 8:35

CutePoison

1414 bronze badges

2

votes

1answer

29 views

How to measure deviance resulting from different random seeds in machine learning?

I'm running an xgboost model to predict probabilities to a binary classification problem. Then I aggregate the results based on the Age variable (what is the aggregated risk of getting the sickness ...

machine-learning xgboost bias

asked Jul 14 '20 at 10:23

user100740

712 bronze badges

2

votes

2answers

38 views

How much can bias decrease performance of the network at the beginnng of the training?

I am writing a custom framework and in it I'm trying to train a simple network to predict the addition function. The network: 1 hidden layer of 3 Neurons 1 output layer cost function used is ...

regression bias

asked Mar 10 '20 at 6:09

David Tóth

1335 bronze badges

2

votes

0answers

77 views

Bias Formula in Machine Learning expanded using ground truth

Why is Bias calculated for $f(x)$? Shouldn't it be calculated for $Y$ (which is $f(x)$ + Noise $\epsilon$)? We are fitting our model to $Y$, So shouldn't we be calculating bias wrt to $Y$? Also, I ...

machine-learning linear-regression machine-learning-model overfitting bias

asked Feb 25 '20 at 20:33

Selvam

696 bronze badges

2

votes

1answer

202 views

LSTM regression bias increases when targets go close to 0

I've build a LSTM model for time series forecasting. Results are not bad, with a mean normalized error of 7%. However, this normalized bias shows a clear pattern: The closer to 0 the value to predict, ...

time-series lstm forecasting bias

asked Nov 29 '18 at 16:19

ignatius

1,4783 silver badges19 bronze badges

2

votes

0answers

23 views

When should the bias b be updated with weights w and when should it be updated seperately?

It seems in some Machine Learning models, the bias term $b$ is updated just like other weights $w_i, i=1...n$. For example, in Logistic Regression, using SGD, $b \ \text{or} \ w_0$ is updated with: $$...

svm logistic-regression optimization bias

asked Jun 18 '18 at 10:43

CyberPlayerOne

3213 silver badges10 bronze badges

1

vote

2answers

188 views

Why is there a trade-off between bias and variance in supervised learning? Why can't we have best of both worlds?

The bias-variance trade-off is like a law in machine learning. You cannot have the best of both worlds. What is it about supervised learning in machine learning that makes it impossible to satisfy the ...

machine-learning supervised-learning variance bias

asked Aug 27 '19 at 14:13

user781486

8951 gold badge8 silver badges16 bronze badges

1

vote

2answers

242 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...

variance bias generalization

asked Jun 15 '18 at 7:33

CyberPlayerOne

3213 silver badges10 bronze badges

1

vote

1answer

62 views

Model biased towards low frequency data?

Generally model gets biased towards data_samples/target whose frequency is high in training data set. Is it possible during training that model gets biased towards low frequency training data set.

training bias

asked May 24 '19 at 2:59

vipin bansal

1,3223 silver badges13 bronze badges

1

vote

1answer

33 views

Training dataset decreasing in quality (Google data science blog)

I have a complex algorithm that decides when it should show customers of an only shop an ad on our website, after they log in, in hope that they will buy what is in the ad. We have no control what is ...

dataset training bias

asked Oct 13 '18 at 18:50

MyCatsHat

1134 bronze badges

1

vote

1answer

15 views

Weights and bias' relative to preprocessed X

I am currently using sklearn scale to preprocess my X data before being put into a perceptron - mean/stddev so as to prevent the ...

plotting perceptron bias

asked Mar 14 '18 at 16:24

maxisme

1135 bronze badges

1

vote

1answer

32 views

Model Selection using Bias Variance Trade Off

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...

machine-learning model-selection variance bias

asked Jan 5 at 11:16

Manish

111 bronze badge

1

vote

1answer

14 views

Is there a quantitative way to determine if a class of algorithms tends produce low bias or low variance models?

I understand that some machine learning models tend to be low bias, whereas others tend to be low variance (source). As an example, a linear regression will tend to have low variance error and high ...

machine-learning model-selection variance bias

asked Dec 2 '20 at 14:26

Brian_E

132 bronze badges

1

vote

1answer

29 views

Do non-parametric models always overfit without regularization?

Let's scope this to just classification. It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy ...

machine-learning overfitting regularization bias non-parametric

asked Jul 16 '20 at 1:23

kennysong

1113 bronze badges

1

vote

0answers

76 views

Derivative of Loss wrt bias term

I read this and have an ambiguity. I try to understand well how to calculate the derivative of Loss w.r.t to bias. In this question, we have this definition: ...

loss-function gradient-descent bias derivation

asked Jun 2 '20 at 0:20

Gonzalo Sanchez cano

1111 bronze badge

1

vote

0answers

299 views

How to calculate irreducible error using Bias and Variance for a given machine learning Model?

I am trying to calculate the Bias and Variance for a ML Model. $$ Err(x)=E[(Y−\hat f(x))^2] \\Err(x)=Bias^2+Variance+Irreducible\ Error $$ $\hat f(x)$ is our model $Y$ is the variable we are trying ...

variance bias

asked May 4 '20 at 19:37

Himanshu Dabas

111 bronze badge

1

vote

0answers

14 views

Measuring the bias of a machine learning model

How can we measure the bias of a machine learning model? Can we determine it by just calculating its performance estimates difference on the train data and test data? For example, if a model SVM ...

machine-learning predictive-modeling bias

asked Apr 13 '20 at 15:16

AAA

111 bronze badge

1

vote

0answers

335 views

Calculation of Neural network biases in backpropagation

While learning neural networks I've found a basic Python working example to play with. It has 3 input nodes, 4 nodes in a hidden layer, 1 output node. 5 data sets for training. The initial code is ...

python neural-network bias

asked Nov 19 '19 at 21:24

Alexey

1111 bronze badge

1

vote

1answer

30 views

What is the defining Set in NLP

I am reading the paper Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings here is the pdf. On page 6, we read: ...

word-embeddings bias

asked Oct 9 '19 at 15:46

chikitin

1435 bronze badges

1

vote

2answers

111 views

bias variance decomposition for classification problem

It is given that: MSE = bias$^2$ + variance I can see the mathematical relationship between MSE, bias, and variance. However, how do we understand the mathematical intuition of bias and variance for ...

classification variance math bias

asked Jun 19 '19 at 12:59

IamTheRealFord

817 bronze badges

1

vote

0answers

108 views

Math behind, MSE = bias^2 + variance

Based on the deeplearningbook: $$MSE = E[(\theta_m^{-} - \theta)^2]$$ $$equals$$ $$Bias(\theta_m^{-})^2 + Var(\theta_m^{-})$$ where m is the number of samples ...

variance bias estimators mse

asked Jun 14 '19 at 5:22

Fatemeh Asgarinejad

1,1243 silver badges15 bronze badges

1

vote

0answers

299 views

Simulate Biased dataset Python [closed]

I am trying simulate data from a normal distribution but bias the sample by excluding all negative values and values divisible by 5 . To demonstrate the effect of bias . I will probably calculate ...

machine-learning bigdata bias

asked Feb 15 '19 at 18:39

DrYoms

112 bronze badges

1

vote

1answer

318 views

Free parameters in logistic regression

When applying logistic regression, one is essentially applying the following function $1/(1 + e^{\beta x})$ to provide a decision boundary, where $\beta$ are a set of parameters that are learned by ...

logistic-regression hyperparameter-tuning bias

asked Oct 13 '18 at 20:29

Mathews24

1758 bronze badges

1

vote

0answers

199 views

Bias of 1 in fully connected layers introduced dying relu problem

While implementing AlexNet (model-code), one of the thing I need to do was to initialize the biases of the convolutional layers and fully connected layers. Normally we initialize biases with 0s, but ...

machine-learning deep-learning convolution convolutional-neural-network bias

asked Aug 22 '18 at 17:15

Abhisek

1114 bronze badges

0

votes

2answers

4k views

Relationship between train and test error

I have some specific questions for which I could not extract answers from books. Therefore, I ask for help here and shall be extremely grateful for an intuitive explanation if possible. In general, ...

machine-learning cnn svm variance bias

asked Oct 4 '18 at 0:48

Ria George

1151 silver badge7 bronze badges

0

votes

1answer

29 views

Bias variance tradeoff boosting (xgboost) vs random forest (randomized bagging) which to use when?

I was looking up differences between boosting an bagging and I see this quoted everywhere If the classifier is unstable (high variance), then we should apply Bagging. If the classifier is stable and ...

random-forest xgboost bias

asked Oct 27 '20 at 16:19

AJITH SHENOY

1

0

votes

1answer

24 views

Does training of neural networks follow the same order in each epoch?

Each epoch uses the weight from the end of the previous epoch(correct me if I am wrong). Is the updating of parameters after each batch always in the same order? To rephrase, are the batches always in ...

deep-learning overfitting bias weight-initialization epochs

asked Jul 24 '20 at 8:47

Borut Flis

1453 bronze badges

0

votes

1answer

25 views

build biased image dataset for emotion analysis

This is a pre-project question. I would like to find or build a biased dataset to demonstrate what happens if training data are biased (biased distributed ethnicity for exemple). I try this for the ...

image-classification bias azure-ml

asked Jun 9 '20 at 13:35

Josselin Tobelem

12 bronze badges

0

votes

1answer

43 views

Should bias updates be porportional to overfitting?

According to questions on the internet, the bias is a learnable parameter, and there are different solutions to updating it, but I failed to find a concise methodology of correctly updating biases ...

cross-validation gradient-descent overfitting bias

asked Mar 11 '20 at 7:23

David Tóth

1335 bronze badges

0

votes

1answer

154 views

Maths question on mean squared error being dervied to bias and variance

I am reading a book and have difficulty in understanding the math on bias- variance tradeoff. Below is the section that I am having trouble with: Given a set of training samples $x_1, x_2, ..., x_n$...

variance bias

asked Jul 28 '18 at 5:39

Carch

132 bronze badges

0

votes

1answer

88 views

Can You Purposely Bias A Clustering Model?

We have a large amount (Billions) of high cardinality, mixed nominal & numerical data, and are performing some clustering on it as an experiment. There is a small subset of these data, however, ...

clustering unsupervised-learning semi-supervised-learning bias

asked Apr 5 '18 at 16:55

TheProletariat

1516 bronze badges

0

votes

0answers

20 views

how to know if there is a bias in data collection methods

I am collecting data for machine learning models I want to build for some application. I started with random sampling (just simply collecting 'recent' data) but I am not getting enough records of ...

data sampling bias

asked Dec 31 '20 at 18:42

Hiro Nakagame

356 bronze badges

0

votes

0answers

18 views

Effect of batch during prediction

During prediction (not training), is it normal to get different loss for different batch size? Worst case happens when I use batch_size=1 for test dataset. The prediction performance get pretty bad. ...

deep-learning prediction bias

asked Nov 2 '20 at 20:03

sovon

4211 gold badge3 silver badges7 bronze badges

Questions tagged [bias]

Related Tags