Questions tagged [bias]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
15 views

Calculation of Neural network biases in backpropagation

While learning neural networks I've found a basic Python working example to play with. It has 3 input nodes, 4 nodes in a hidden layer, 1 output node. 5 data sets for training. The initial code is ...
4
votes
2answers
81 views

Bagging vs Boosting, Bias vs Variance, Depth of trees

I understand the main principle of bagging and boosting for classification and regression trees. My doubts are about the optimization of the hyperparameters, especially the depth of the trees First ...
0
votes
0answers
9 views

Reflections about types of bias in comparative analysis

I am working on an analysis on proposal rejections across two different periods in time and I was wondering what reflections I would have to make about the risk of bias. The reason for this is ...
0
votes
1answer
14 views

What is the defining Set in NLP

I am reading the paper Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings here is the pdf. On page 6, we read: ...
0
votes
0answers
7 views

Derive main prediction for zero-one loss function

Intuitively I can see why mode of predictions is the main prediction of a zero-one loss function, but mathematically I am not sure how it is derived? Main prediction $= argmin_{y'}E_D[L(\widehat y, y'...
3
votes
0answers
93 views

Learning curve using micro F-score and macro F-score

I plotted the learning curves using micro and macro F-scores for a Multinomial Naive Bayes classifier. The first plot is made using micro F-score, and the second using macro F-score. I find it quite ...
1
vote
2answers
100 views

Why is there a trade-off between bias and variance in supervised learning? Why can't we have best of both worlds?

The bias-variance trade-off is like a law in machine learning. You cannot have the best of both worlds. What is it about supervised learning in machine learning that makes it impossible to satisfy the ...
0
votes
0answers
28 views

Working with a biased sample of the data

I have a project to build an automatic fraud detection system for an energy company. I have a huge amount of unlabeled data and only an small and biased labeled one. My labeled data is based on ...
1
vote
0answers
30 views

I have tried 5 different types of model but all returns really low training accuracy (~64%) and low testing accuracy (~14%). What should I do?

I am working with a typical regressor problem. There are $6$ features in the dataset that I am concerned with. There are about $800$ data points in my dataset. The features and the predicted values ...
0
votes
1answer
33 views

Bias and variance in the model o in the predictions?

This topic confuses me. In the literature or articles, when talking about bias and variance in automatic learning, specifically in cross-validation, do they refer to the high bias (underfitting) and ...
1
vote
1answer
71 views

bias variance decomposition for classification problem

It is given that: MSE = bias$^2$ + variance I can see the mathematical relationship between MSE, bias, and variance. However, how do we understand the mathematical intuition of bias and variance for ...
1
vote
0answers
56 views

Math behind, MSE = bias^2 + variance

Based on the deeplearningbook: $$MSE = E[(\theta_m^{-} - \theta)^2]$$ $$equals$$ $$Bias(\theta_m^{-})^2 + Var(\theta_m^{-})$$ where m is the number of samples ...
1
vote
1answer
19 views

Model biased towards low frequency data?

Generally model gets biased towards data_samples/target whose frequency is high in training data set. Is it possible during training that model gets biased towards low frequency training data set.
2
votes
1answer
92 views

Bias-variance tradeoff and the uncertainty principle

Bias variance tradeoff seems to behave like the uncertainty principle, is it just another name for the same principle?
2
votes
1answer
149 views

Is normalizing the validation set of time series a kind of look ahead bias?

Here's the data normalization process of a time series in a paper about stock prediction using LSTM: Split train and test set based on time (e.g. training set: 2001-2010, test set:2011-2012). This ...
0
votes
0answers
58 views

How can I recognise if I can improve a random forest model by adding features

I want to tune a random forest model with caret package. I'm tuning it with cross-validation to prevent overfitting and resulted cross-validation accuracy is very ...
1
vote
0answers
178 views

Simulate Biased dataset Python [closed]

I am trying simulate data from a normal distribution but bias the sample by excluding all negative values and values divisible by 5 . To demonstrate the effect of bias . I will probably calculate ...
1
vote
3answers
1k views

Why underfitting is called high bias and overfitting is called high variance?

I have been using terms like underfitting/overfitting and bias-variance tradeoff for quite some while in data science discussions and I understand that underfitting is associated with high bias and ...
3
votes
1answer
188 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
2
votes
3answers
264 views

Linear machine learning algorithms “often” have high bias/low variance?

In this blog, which explains the meaning of bias and variance in machine learning, there's a line under the heading "Bias-Variance Trade-Off" which says: Parametric or linear machine learning ...
2
votes
1answer
44 views

svm optimization problem

Suppose we have the dataset: {(3,1),(3-1),(6,1),(6,-1)} {(1,0),(0,1),(0,-1),(-1,0)} the first set represent the positive label, and de second the negative. I want manually find the support vectors, ...
1
vote
1answer
131 views

LSTM regression bias increases when targets go close to 0

I've build a LSTM model for time series forecasting. Results are not bad, with a mean normalized error of 7%. However, this normalized bias shows a clear pattern: The closer to 0 the value to predict, ...
1
vote
1answer
208 views

Free parameters in logistic regression

When applying logistic regression, one is essentially applying the following function $1/(1 + e^{\beta x})$ to provide a decision boundary, where $\beta$ are a set of parameters that are learned by ...
1
vote
1answer
31 views

Training dataset decreasing in quality (Google data science blog)

I have a complex algorithm that decides when it should show customers of an only shop an ad on our website, after they log in, in hope that they will buy what is in the ad. We have no control what is ...
0
votes
2answers
1k views

Relationship between train and test error

I have some specific questions for which I could not extract answers from books. Therefore, I ask for help here and shall be extremely grateful for an intuitive explanation if possible. In general, ...
1
vote
0answers
109 views

Bias of 1 in fully connected layers introduced dying relu problem

While implementing AlexNet (model-code), one of the thing I need to do was to initialize the biases of the convolutional layers and fully connected layers. Normally we initialize biases with 0s, but ...
0
votes
1answer
75 views

Maths question on mean squared error being dervied to bias and variance

I am reading a book and have difficulty in understanding the math on bias- variance tradeoff. Below is the section that I am having trouble with: Given a set of training samples $x_1, x_2, ..., x_n$...
1
vote
0answers
19 views

When should the bias b be updated with weights w and when should it be updated seperately?

It seems in some Machine Learning models, the bias term $b$ is updated just like other weights $w_i, i=1...n$. For example, in Logistic Regression, using SGD, $b \ \text{or} \ w_0$ is updated with: $$...
1
vote
2answers
211 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
5
votes
2answers
100 views

Trade off between Bias and Variance [closed]

What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.
9
votes
1answer
560 views

Question on bias-variance tradeoff and means of optimization

So I was wondering how does one, for example, can best optimize the model they are trying to build when confronted with issues presented by high bias or high variance. Now, of course, you can play ...
0
votes
1answer
59 views

Can You Purposely Bias A Clustering Model?

We have a large amount (Billions) of high cardinality, mixed nominal & numerical data, and are performing some clustering on it as an experiment. There is a small subset of these data, however, ...
1
vote
1answer
15 views

Weights and bias' relative to preprocessed X

I am currently using sklearn scale to preprocess my X data before being put into a perceptron - mean/stddev so as to prevent the ...
3
votes
1answer
405 views

Correcting log-bias in the output of an XGB

I have previously worked with GAMs, where I was trying to do regression on a log-transformed variable. The log-transformation introduced a negative bias in the average of the predicted variable, and I ...
0
votes
0answers
307 views

Why bias value is not significant after normalizing inputs?

In my textbook, I read that whenever you reduce the mean of each feature from corresponding features in the training data and divide each feature by its standard deviation (this process is called ...