Questions tagged [bias]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
1answer
29 views

Do non-parametric models always overfit without regularization?

Let's scope this to just classification. It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy ...
0
votes
1answer
33 views

Amount of data needed for deep learning vs support vector machine

I often read about the fact, that the amount of data to train and get a generalizing model for a deep learning algorithm is much higher in comparison, e.g. to a support vector machine. It makes sense, ...
1
vote
0answers
199 views

Bias of 1 in fully connected layers introduced dying relu problem

While implementing AlexNet (model-code), one of the thing I need to do was to initialize the biases of the convolutional layers and fully connected layers. Normally we initialize biases with 0s, but ...
1
vote
1answer
32 views

Model Selection using Bias Variance Trade Off

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...
3
votes
1answer
87 views

Bagging vs pasting in ensemble learning

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is ...
0
votes
0answers
20 views

how to know if there is a bias in data collection methods

I am collecting data for machine learning models I want to build for some application. I started with random sampling (just simply collecting 'recent' data) but I am not getting enough records of ...
0
votes
2answers
48 views

Bias and variance in the model o in the predictions?

This topic confuses me. In the literature or articles, when talking about bias and variance in automatic learning, specifically in cross-validation, do they refer to the high bias (underfitting) and ...
5
votes
2answers
775 views

What are bias and variance in machine learning?

I am studying machine learning, and I have encountered the concept of bias and variance. I am a university student and in the slides of my professor, the bias is defined as: $bias = E[error_s(h)]-...
1
vote
2answers
111 views

bias variance decomposition for classification problem

It is given that: MSE = bias$^2$ + variance I can see the mathematical relationship between MSE, bias, and variance. However, how do we understand the mathematical intuition of bias and variance for ...
3
votes
1answer
34 views

Whether add bias or not in a perceptron

In some places, perceptron is described as having added bias, while in some places, bias is not added. Which one is right for you?
2
votes
1answer
29 views

How to measure deviance resulting from different random seeds in machine learning?

I'm running an xgboost model to predict probabilities to a binary classification problem. Then I aggregate the results based on the Age variable (what is the aggregated risk of getting the sickness ...
1
vote
1answer
14 views

Is there a quantitative way to determine if a class of algorithms tends produce low bias or low variance models?

I understand that some machine learning models tend to be low bias, whereas others tend to be low variance (source). As an example, a linear regression will tend to have low variance error and high ...
20
votes
5answers
9k views

Why underfitting is called high bias and overfitting is called high variance?

I have been using terms like underfitting/overfitting and bias-variance tradeoff for quite some while in data science discussions and I understand that underfitting is associated with high bias and ...
3
votes
1answer
287 views

Learning curve using micro F-score and macro F-score

I plotted the learning curves using micro and macro F-scores for a Multinomial Naive Bayes classifier. The first plot is made using micro F-score, and the second using macro F-score. I find it quite ...
1
vote
1answer
30 views

What is the defining Set in NLP

I am reading the paper Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings here is the pdf. On page 6, we read: ...
0
votes
0answers
18 views

Effect of batch during prediction

During prediction (not training), is it normal to get different loss for different batch size? Worst case happens when I use batch_size=1 for test dataset. The prediction performance get pretty bad. ...
0
votes
1answer
29 views

Bias variance tradeoff boosting (xgboost) vs random forest (randomized bagging) which to use when?

I was looking up differences between boosting an bagging and I see this quoted everywhere If the classifier is unstable (high variance), then we should apply Bagging. If the classifier is stable and ...
2
votes
2answers
71 views

Predictive modeling when output affects future input

Assume I have a model which predicts the outcome of number of icecream sold in a store. The model is trained on data for the last 5 years while keep the last year as a validation set and has produced ...
0
votes
0answers
10 views

Dropping Missing Observations under MAR Assumption

Some of the outcome data in my data set are missing. I believe that the missing data mechanism is missing at random (MAR) as the observed characteristics significantly differ between the missing and ...
2
votes
1answer
45 views

Why is it okay to set the bias vector up with zeros, and not the weight matrices?

We do not initialize weight matrices with zeros because the symmetry isn’t broken during the backward pass, and subsequently in the parameter updating process. But it is safe to set the bias vector up ...
2
votes
4answers
1k views

Linear machine learning algorithms “often” have high bias/low variance?

In this blog, which explains the meaning of bias and variance in machine learning, there's a line under the heading "Bias-Variance Trade-Off" which says: Parametric or linear machine learning ...
0
votes
0answers
32 views

Odd results on bias-variance tradeoff assessment

I am running a bias-variance tradeoff assessment on ten regression models of increasing complexity (linear to x^10), but my results do not satisfy the MSE = Bias^2 + Variance + True Error relationship....
1
vote
0answers
76 views

Derivative of Loss wrt bias term

I read this and have an ambiguity. I try to understand well how to calculate the derivative of Loss w.r.t to bias. In this question, we have this definition: ...
0
votes
0answers
19 views

DNN predicting the same value for train+test Data

I have trained a Deep Neural architecture for regression problem and after the hundred's of epochs, model predicting the same output for both training and testing data. When I reduced the batch size, ...
0
votes
1answer
25 views

build biased image dataset for emotion analysis

This is a pre-project question. I would like to find or build a biased dataset to demonstrate what happens if training data are biased (biased distributed ethnicity for exemple). I try this for the ...
2
votes
1answer
214 views

How does C have effects on bias and variance of a Support Vector Machine?

The minimization problem for SVM can be written as- $$\overset{\text{min}}{\theta} C\sum_{i = 1}^{m}{[y^icost_1(\theta^Tx^i) + (1-y^i)cost_0(\theta^Tx^i)]} + \frac12\sum_{j = 1}^n{\theta_j}^2$$ Now, ...
0
votes
1answer
24 views

Does training of neural networks follow the same order in each epoch?

Each epoch uses the weight from the end of the previous epoch(correct me if I am wrong). Is the updating of parameters after each batch always in the same order? To rephrase, are the batches always in ...
0
votes
0answers
16 views

Is recommendable look for high variance when your data is imbalance?

Hello I have a dataset with the following classes A, B, C, and this classes have the following representation of the dataset 60% 39% 1%. Is it a good idea try to get a model with high variance in this ...
2
votes
1answer
52 views

Which between random forest or extra tree is best in a unbalance dataset?

I have an unbalanced dataset, with 3 classes, with 60% of class 1, 38% of class 2, and 2% of class 3. I don't want to generate more examples of class 3, and I cannot get more examples of class 3. The ...
0
votes
0answers
19 views

Question regarding strata in Geron's book

In the book "Hands-On Machine Learning with Scikit-Learn and Tensorflow" by Aurélien Géron. There is a regression project explained. My doubt is regarding his example for 'stratified sampling'. He ...
3
votes
1answer
394 views

Is normalizing the validation set of time series a kind of look ahead bias?

Here's the data normalization process of a time series in a paper about stock prediction using LSTM: Split train and test set based on time (e.g. training set: 2001-2010, test set:2011-2012). This ...
0
votes
0answers
30 views

Exploratory statistics, how to idenify and remove driver (bias)

I am looking at customer data, and created frequency tables (+histograms) for customers with different professional statuses and what the best time is to reach them. Status ranges here from employed, ...
0
votes
0answers
18 views

How to reconstruct an unbiased data set?

Imagine that every month, I have a "virgin" data set consisting of data points (e.g. people that have stopped paying a subscription) with certain features (e.g. geo-demographic information and payment ...
2
votes
1answer
202 views

LSTM regression bias increases when targets go close to 0

I've build a LSTM model for time series forecasting. Results are not bad, with a mean normalized error of 7%. However, this normalized bias shows a clear pattern: The closer to 0 the value to predict, ...
1
vote
0answers
299 views

How to calculate irreducible error using Bias and Variance for a given machine learning Model?

I am trying to calculate the Bias and Variance for a ML Model. $$ Err(x)=E[(Y−\hat f(x))^2] \\Err(x)=Bias^2+Variance+Irreducible\ Error $$ $\hat f(x)$ is our model $Y$ is the variable we are trying ...
1
vote
0answers
14 views

Measuring the bias of a machine learning model

How can we measure the bias of a machine learning model? Can we determine it by just calculating its performance estimates difference on the train data and test data? For example, if a model SVM ...
2
votes
2answers
38 views

How much can bias decrease performance of the network at the beginnng of the training?

I am writing a custom framework and in it I'm trying to train a simple network to predict the addition function. The network: 1 hidden layer of 3 Neurons 1 output layer cost function used is ...
0
votes
1answer
43 views

Should bias updates be porportional to overfitting?

According to questions on the internet, the bias is a learnable parameter, and there are different solutions to updating it, but I failed to find a concise methodology of correctly updating biases ...
2
votes
0answers
77 views

Bias Formula in Machine Learning expanded using ground truth

Why is Bias calculated for $f(x)$? Shouldn't it be calculated for $Y$ (which is $f(x)$ + Noise $\epsilon$)? We are fitting our model to $Y$, So shouldn't we be calculating bias wrt to $Y$? Also, I ...
1
vote
2answers
188 views

Why is there a trade-off between bias and variance in supervised learning? Why can't we have best of both worlds?

The bias-variance trade-off is like a law in machine learning. You cannot have the best of both worlds. What is it about supervised learning in machine learning that makes it impossible to satisfy the ...
1
vote
0answers
335 views

Calculation of Neural network biases in backpropagation

While learning neural networks I've found a basic Python working example to play with. It has 3 input nodes, 4 nodes in a hidden layer, 1 output node. 5 data sets for training. The initial code is ...
5
votes
2answers
534 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
0
votes
0answers
35 views

I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?

I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
1
vote
2answers
242 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
1
vote
0answers
108 views

Math behind, MSE = bias^2 + variance

Based on the deeplearningbook: $$MSE = E[(\theta_m^{-} - \theta)^2]$$ $$equals$$ $$Bias(\theta_m^{-})^2 + Var(\theta_m^{-})$$ where m is the number of samples ...
1
vote
1answer
62 views

Model biased towards low frequency data?

Generally model gets biased towards data_samples/target whose frequency is high in training data set. Is it possible during training that model gets biased towards low frequency training data set.
2
votes
1answer
225 views

Bias-variance tradeoff and the uncertainty principle

Bias variance tradeoff seems to behave like the uncertainty principle, is it just another name for the same principle?
1
vote
0answers
299 views

Simulate Biased dataset Python [closed]

I am trying simulate data from a normal distribution but bias the sample by excluding all negative values and values divisible by 5 . To demonstrate the effect of bias . I will probably calculate ...
3
votes
1answer
463 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
3
votes
1answer
600 views

Correcting log-bias in the output of an XGB

I have previously worked with GAMs, where I was trying to do regression on a log-transformed variable. The log-transformation introduced a negative bias in the average of the predicted variable, and I ...