Questions tagged [bias]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Learning high bias in neural net

I have this simple model, which tries to predict constant [1, 1, .. 1, 0, ..., 0] vector regardless of input. I found that model predicts it successfully if trained on input in [0,10] range, however ...
user avatar
1 vote
0 answers
8 views

how to test if the target variables is correlated with protected variables?

I wonder how to check if the protected variables in fairness either encoded in the other features (non-protected). Or if they are not sufficiently correlated with target variables so adding them does ...
user avatar
  • 13
0 votes
1 answer
23 views

How to provide Intentional Bias towards recent examples in Text Classification?

I have trained an XGBClassifier to classify text issues to a rightful assignee (simple 50-way classification). The source from where I am fetching the data also provides a datetime object which gives ...
user avatar
1 vote
0 answers
20 views

Do these values of bias and variance make sense?

I have this code: ...
user avatar
0 votes
0 answers
7 views

Handling bias inputs during normalization

Suppose I have an input matrix $\mathbf X\in \mathbb R^{(D+1)\times N}$ where $N$ is number of samples $D$ is dimension of an input vector $x$ and extra $1$ dimension is for bias where all bias ...
user avatar
0 votes
0 answers
19 views

How to manage sampling bias between training data and real-world data?

I'm currently working on a binary classification problem. My training dataset is rather small with only 1000 elements. (I don't know if it is relevant : my problem is similar to the "spam ...
user avatar
  • 1
0 votes
1 answer
28 views

Look ahead bias predicting a time series using features

I am making some ML methods (RF, RNN, MLP) to predict a time series value 'y' based on features 'X' and not the time series 'y' itself. My question is regarding the bias I might be including since I ...
user avatar
  • 3
0 votes
1 answer
101 views

Backpropagation of Bias in Neural networks

My goal is to calculate backpropagation(Especially the backpropagation of the bias). For example, X, W and B are python numpy array, such as [[0,0],[0,1]] , [[5,5,5],[10,10,10]] and [1,2,3] for each. ...
user avatar
  • 105
2 votes
1 answer
30 views

learning curves of a classification algorithm

I a trying to understand this learning curve of a classification problem. But I am not sure what to infer. I believe that I have overfitting but I cannot sure. Very low training loss that’s very ...
user avatar
  • 21
0 votes
1 answer
32 views

Data snooping and information leakage?

I need help in deciding whether my below implementation imposes data snooping bias and information leakage from the test/evaluation set to the train set. I have a text corpus of 10k+ short online ...
user avatar
  • 257
1 vote
1 answer
42 views

The behavior of the cross validation error and training error in underfitting case is not clear

I currently study the "Machine Learning" course on Coursera.org by Andrew Ng, it comes to a topic that discusses the performance of learning algorithms under different conditions. Here, we ...
user avatar
1 vote
0 answers
86 views

Keras model prediction always has unwanted offset

I am trying to predict next 10 days by looking into the last 60 days. So tried to implement an LSTM layer. Before jumping into the question, I want to clarify a few points. Firstly, this is a Multiple ...
user avatar
  • 61
0 votes
0 answers
16 views

Adding $1$ feature to data point to avoid bias

Given data point $x\in X,\ x\in \mathbb{R}^p$, once we resolve the parameters of the linear discriminant model, we will have $\hat{B} = (X^TX)^{-1}X^TY$, where $Y \in \mathbb{R}^{N\times K}$ is the ...
user avatar
  • 179
0 votes
0 answers
77 views

Does Bias always decrease when Complexity increase?

(I'm just starting learning about ML stuff and so please don't be rude if the following question is to stupid or totally wrong) I'm reading about Bias-Variance Trade off and I don't understand the (...
user avatar
1 vote
0 answers
10 views

Fractional Differencing/Differentiation for Non-Time based Model; Look-ahead bias?

I have time-series data, but instead of using a time-based model like RNN, I've decided to approach my classification problem using an lgbm classifier. To do so, I have modified the data, such that ...
user avatar
0 votes
1 answer
13 views

I am not sure whether I am being asked to calculate the variance or the irreducible error

The question: Suppose we randomly sample a training set D from some unknown distribution. For each training set D we sample, we train a regression model to predict $y$ from $x$. We repeat this 10 ...
user avatar
1 vote
1 answer
29 views

Does class weighting encourage overfitting when the true class distribution is imbalanced?

I am working on a classification problem in which ~90% of samples come from class 1 while ~10% of samples come from class 2. I have been using various techniques to combat the class imbalance while ...
user avatar
1 vote
0 answers
25 views

Checking my understanding of the bias-variance tradeoff

Hello fellow DS aficionados, I'm trying to clarify some confusing feedback I got from a homework problem. We were told to recreate the bias-variance tradeoff graph using the first graph below as an ...
user avatar
1 vote
1 answer
27 views

Explanation of Karpathy tweet about common mistakes. #5: "you didn't use bias=False for your Linear/Conv2d layer when using BatchNorm"

I recently found this twitter thread from Andrej Karpathy. In it he states a few common mistakes during the development of a neural network. you didn't try to overfit a single batch first. you forgot ...
user avatar
  • 123
0 votes
1 answer
138 views

Visualizing the equation for separating hyperplane

I was wondering if I can visualize with the example the fact that for all points $x$ on the separating hyperplane, the following equation holds true: $$w^T.x+w_0=0\quad\quad\quad \text{... equation (1)...
user avatar
  • 165
0 votes
0 answers
26 views

What is correct equation for LR decision boundary?

I read that the equation perceptron decision boundary is given as follows:$$w^Tx-w_0=0$$ This can be proven as follows: Assuming $w$ is a unit vector (as we can multiply above equation with a ...
user avatar
  • 165
0 votes
0 answers
20 views

Is it always that lower tree with higher bias but higher tree with higher varaince

When dealing with bias and variance trade-offs, I always hear that in tree models: shallow tree = high bias but low variance, deep tree = low bias but high variance. Someone may also quote from high ...
user avatar
0 votes
0 answers
30 views

How to measure (and get rid of) biases in training data?

There might be correlation in the training data that do not occur in 'nature' (or occur to a much smaller extent, or maybe we are legally required to not look at these correlations). In my concrete ...
user avatar
  • 101
1 vote
4 answers
212 views

What do "Under fitting" and "Over fitting" really mean? They have never been clearly defined

I am always getting lost when dealing with these terms. Especially being asked questions about the relationship such as underfitting-high bias (low variance) or overfitting-high variance (low bias). ...
user avatar
0 votes
1 answer
38 views

How to introduce bias in a machine learning model?

How can I introduce bias for a decision tree model while building an ML application? e.g. If I am building a stock trading recommendation algorithim, I would want to recommend a stock only when the ...
user avatar
  • 83
3 votes
3 answers
117 views

Difference between ethics and bias in Machine Learning

I'm confused about the difference between "ethics" and "bias" when those concepts are discussed in the context of Machine Learning (ML). In my understanding, ethical issue in ML is ...
user avatar
  • 31
0 votes
3 answers
141 views

Imbalanced classification with bias

The problem: A business historical heuristic rule for offering a special deal to customers has created a bias in the dataset when trying to use machine learning in order to make a more sophisticated ...
user avatar
  • 41
0 votes
0 answers
14 views

Effect Analysis with Survival Bias Data

First of all, let me define some terms: Survival Bias The data we collected is biased (because we only managed to recognise users who survived through the process) Landing Page The first website page ...
user avatar
0 votes
0 answers
29 views

How to retrain a model without bias

Let's say I have a model in production that predicts a certain customer behavior (for example, propensity to buy), and let's say that there is a business process underneath that takes some actions ...
user avatar
1 vote
0 answers
11 views

Generalization error problem on training set

Training data: $\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$ you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
user avatar
0 votes
1 answer
33 views

KNN Variance using a high value of K and cross-validation

it has come to my understanding, that a value of K=1, gives a high variance because we are only using only one data point, hence we are very likely to model the noise in that training example. Bias: ...
user avatar
1 vote
0 answers
25 views

Active learning with mixture model cluster assignments - am I injecting bias here?

Suppose I have a dataset of people's phone numbers and heights, and I'm interested in learning the parameters $p_{girl}$, $p_{boy}=1-p_{girl}$, $\mu_{boy}$, $\mu_{girl}$, and overall $\sigma$ ...
user avatar
  • 76
3 votes
0 answers
133 views

Unbiasedness of random forests

Suppose that I am trying to build a random forest by subsampling the data and choosing a single feature per tree randomly. For example, suppose there is some dataset, $D = \{(x_{1},y_{1}), ......(x_{N}...
user avatar
  • 131
1 vote
1 answer
91 views

Mathematical bias and weight vs machine learning bias and weight

I am a little confused about the term Bias and Weight with respect to machine learning. Say we want to predict the heights of people whose weights are given. So plot weights to x-axis and height to ...
user avatar
  • 137
0 votes
1 answer
196 views

Neural network: does bias equal to zero, is the same as, a layer without bias?

Question as in the title. Does bias equal to zero, is the same as, removing bias from the layer? Here's a pytorch implementation to showcase what I mean. ...
user avatar
  • 103
1 vote
1 answer
49 views

Model Selection using Bias Variance Trade Off

I have a Regression Model with Train MAPE as 6% and Test MAPE as 15%. This appears to me as a clear case of over fitting. But can I still use this model assuming 15% Error is not a bad number after-...
user avatar
  • 11
0 votes
0 answers
27 views

how to know if there is a bias in data collection methods

I am collecting data for machine learning models I want to build for some application. I started with random sampling (just simply collecting 'recent' data) but I am not getting enough records of ...
user avatar
1 vote
1 answer
26 views

Is there a quantitative way to determine if a class of algorithms tends produce low bias or low variance models?

I understand that some machine learning models tend to be low bias, whereas others tend to be low variance (source). As an example, a linear regression will tend to have low variance error and high ...
user avatar
  • 13
0 votes
1 answer
252 views

Bias variance tradeoff boosting (xgboost) vs random forest (randomized bagging) which to use when?

I was looking up differences between boosting an bagging and I see this quoted everywhere If the classifier is unstable (high variance), then we should apply Bagging. If the classifier is stable and ...
user avatar
1 vote
1 answer
17 views

Dropping Missing Observations under MAR Assumption

Some of the outcome data in my data set are missing. I believe that the missing data mechanism is missing at random (MAR) as the observed characteristics significantly differ between the missing and ...
user avatar
  • 11
2 votes
1 answer
239 views

Why is it okay to set the bias vector up with zeros, and not the weight matrices?

We do not initialize weight matrices with zeros because the symmetry isn’t broken during the backward pass, and subsequently in the parameter updating process. But it is safe to set the bias vector up ...
user avatar
  • 180
3 votes
2 answers
83 views

Predictive modeling when output affects future input

Assume I have a model which predicts the outcome of the number of icecreams sold in a store. The model is trained on data for the last 5 years while keeping the last year as a validation set and has ...
user avatar
0 votes
0 answers
35 views

Odd results on bias-variance tradeoff assessment

I am running a bias-variance tradeoff assessment on ten regression models of increasing complexity (linear to x^10), but my results do not satisfy the MSE = Bias^2 + Variance + True Error relationship....
user avatar
  • 21
2 votes
1 answer
364 views

How does C have effects on bias and variance of a Support Vector Machine?

The minimization problem for SVM can be written as- $$\overset{\text{min}}{\theta} C\sum_{i = 1}^{m}{[y^icost_1(\theta^Tx^i) + (1-y^i)cost_0(\theta^Tx^i)]} + \frac12\sum_{j = 1}^n{\theta_j}^2$$ Now, ...
user avatar
  • 180
6 votes
3 answers
2k views

What are bias and variance in machine learning?

I am studying machine learning, and I have encountered the concept of bias and variance. I am a university student and in the slides of my professor, the bias is defined as: $bias = E[error_s(h)]-...
user avatar
  • 721
0 votes
1 answer
48 views

Does training of neural networks follow the same order in each epoch?

Each epoch uses the weight from the end of the previous epoch(correct me if I am wrong). Is the updating of parameters after each batch always in the same order? To rephrase, are the batches always in ...
user avatar
1 vote
1 answer
138 views

Do non-parametric models always overfit without regularization?

Let's scope this to just classification. It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy ...
user avatar
  • 111
3 votes
1 answer
45 views

How to measure deviance resulting from different random seeds in machine learning?

I'm running an xgboost model to predict probabilities to a binary classification problem. Then I aggregate the results based on the Age variable (what is the aggregated risk of getting the sickness ...
user avatar
2 votes
1 answer
107 views

Which between random forest or extra tree is best in a unbalance dataset?

I have an unbalanced dataset, with 3 classes, with 60% of class 1, 38% of class 2, and 2% of class 3. I don't want to generate more examples of class 3, and I cannot get more examples of class 3. The ...
user avatar
  • 317
0 votes
1 answer
29 views

build biased image dataset for emotion analysis

This is a pre-project question. I would like to find or build a biased dataset to demonstrate what happens if training data are biased (biased distributed ethnicity for exemple). I try this for the ...
user avatar