Questions tagged [variance]

The tag has no usage guidance.

8
votes
1answer
380 views

Question on bias-variance tradeoff and means of optimization

So I was wondering how does one, for example, can best optimize the model they are trying to build when confronted with issues presented by high bias or high variance. Now, of course, you can play ...
6
votes
3answers
3k views

Overfitting Naive Bayes

My question is what are potential reasons for Naive Bayes to perform well on a train set but poorly on a test set? I am working with a variation of the 20news dataset. The dataset has documents, ...
4
votes
3answers
892 views

What is the meaning of term Variance in Machine Learning Model?

I am familiar with terms high bias and high variance and their effect on the model. Basically your model has high variance when it is too complex and sensitive too even outliers. But recently I was ...
4
votes
2answers
76 views

Trade off between Bias and Variance [closed]

What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.
4
votes
2answers
819 views

How to estimate the variance of regressors in scikit-learn?

Every classifier in scikit-learn has a method predict_proba(x) that predicts class probabilities for x. How to do the same thing ...
4
votes
2answers
942 views

How do you set sigma for the Gaussian similarity kernel?

Let's say we have $n$ two-dimensional vectors: $$\mathbf{x}_1,\dots,\mathbf{x}_i,\dots,\mathbf{x}_n=(x_{1_1},x_{1_2})^T,\dots,(x_{i_1},x_{i_2})^T,\dots,(x_{n_1},x_{n_2})^T$$ How do you set $\sigma$ ...
3
votes
1answer
105 views

Bias-variance tradeoff in practice (CNN)

I first trained a CNN on my dataset and got a loss plot that looks somewhat like this: Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
3
votes
1answer
38 views

Bootstrapping or Randomly Dividing Dataset to reduce variance?

If I have 10,000 training samples then what should I do: Bootstrapping and train 10 classifiers on it and then aggregating Or randomly divide the dataset into 10 parts and train 10 classifiers on ...
2
votes
1answer
35 views

Bias-variance tradeoff and the uncertainty principle

Bias variance tradeoff seems to behave like the uncertainty principle, is it just another name for the same principle?
2
votes
0answers
21 views

Measure of variety within list/cluster

I have a dataset of about 53000 points. It has been clustered twice, based on two sets of unrelated attributes. For the first clustering (clustering 1) I used DBScan, and it ended up with about 700 ...
1
vote
1answer
2k views

RL Advantage function why A = Q-V instead of A=V-Q?

In RL Course by David Silver - Lecture 7: Policy Gradient Methods, David explains what an Advantage function is, and how it's the difference between Q(s,a) and the V(s) Preliminary, from this post: ...
1
vote
2answers
101 views

How can I calculate mean and variance incrementally?

Say I have a set S of values, and want to store in a database some summary information about that set, so that later when I acquire a new value v I can make a reasonable estimate of what the summary ...
1
vote
2answers
64 views

Why can decision trees have a high amount of variance

I've heard that decision trees can have a high amount of variance, and that for a data set $D$ split into test/train the decision tree could be quite different depending on how the data was split. ...
1
vote
2answers
94 views

Linear machine learning algorithms “often” have high bias/low variance?

In this blog, which explains the meaning of bias and variance in machine learning, there's a line under the heading "Bias-Variance Trade-Off" which says: Parametric or linear machine learning ...
1
vote
2answers
186 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
1
vote
1answer
2k views

How to decide what threshold to use for removing low-variance features?

How to decide what threshold to use for removing low-variance features? Particularly, I have 100000 features and the variances look like: Could I e.g. take the average and use it to split this to ~...
1
vote
0answers
32 views

Math behind, MSE = bias^2 + variance

Based on the deeplearningbook: $$MSE = E[(\theta_m^{-} - \theta)^2]$$ $$equals$$ $$Bias(\theta_m^{-})^2 + Var(\theta_m^{-})$$ where m is the number of samples ...
1
vote
0answers
17 views

F-test for comparing the mean of two groups

What would be the procedure, in terms of F-test, if I would like to check if the mean of one group is greater than the other group (alternative hypothesis: $\mu_1 > \mu_2$)? And also what would ...
1
vote
0answers
8 views

Would it make technical sense to model its two known subelements separately (CLV example)?

Let's assume Customer Lifetime Value (CLV) is defined as average basket x frequency. Option A is to build model predicting CLV ...
1
vote
0answers
83 views

Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)

When comparing several regression models in terms of quality, it seems like most have agreed on the MSE. There are also papers comparing "variance" and "variance accounted for (VAF)". However, there ...
1
vote
1answer
24 views

sort occurrence matrix to minimize its spatial variance

Is there any algorithm or an approach that sort occurrence matrix to reduce its spatial variance. I mean by spatial variance moving window variance (n*n moving box).
1
vote
0answers
344 views

How to sort tensor along two axis by variance in TensorFlow?

I have a function that visualizes 2D convolutional kernels. The weights come as 4D-tensors [filter_height, filter_width, in_channels, out_channels], as specified for conv2d. Now I try to sort the ...
1
vote
1answer
73 views

Convert a pdf into a conditional pdf such that mean increases and std dev falls

Let success metric(for some business use case I am working on) be a continuous random variable S. The mean of pdf defined on S indicates the chance of success. Higher the mean more is the chance of ...
0
votes
4answers
90 views

Meaning of variance in machine learning models

I know that high variance cause overfitting, and high variance is that the model is sensitive to outliers. But can I say Variance is that when the predicted points are too prolonged lead to high ...
0
votes
3answers
333 views

Why underfitting is called high bias and overfitting is called high variance?

I have been using terms like underfitting/overfitting and bias-variance tradeoff for quite some while in data science discussions and I understand that underfitting is associated with high bias and ...
0
votes
2answers
514 views

Relationship between train and test error

I have some specific questions for which I could not extract answers from books. Therefore, I ask for help here and shall be extremely grateful for an intuitive explanation if possible. In general, ...
0
votes
1answer
65 views

Maths question on mean squared error being dervied to bias and variance

I am reading a book and have difficulty in understanding the math on bias- variance tradeoff. Below is the section that I am having trouble with: Given a set of training samples $x_1, x_2, ..., x_n$...
0
votes
1answer
53 views

very low variance explained after applying pca

I applied PCA on MNIST data and found that the first 64 components are able to retain 86% of variance. Is there any problem while applying pca to a big dataset like MNIST. Because in most of the ...
0
votes
1answer
434 views

How to measure variance in a classification dataset?

I have a dataset that contains 20 predictor variables (both categorical and numeric) and one target variable (2 classes - Good and Bad). But, there are only 23 observations in the dataset. While I ...
0
votes
0answers
14 views

ZCA: Covariance of Large Image Database

I have an overall question if my method is sound, so please bear with me in this description :). I have a large image database and I wanted to create a preprocessing step using ZCA. The issue is that ...
0
votes
1answer
37 views

Massive variation in results with tensorflow and keras

I'm new to Tensorflow and Keras and I some background knowledge of how CNN's work. I'm using a basic sequential model based on the code by https://pythonprogramming.net/convolutional-neural-network-...
0
votes
0answers
7 views

Two tailed F test, correct rejection of the null hypothesis

I am performing an F test for equality of two variances by following this website formula https://itl.nist.gov/div898/handbook/eda/section3/eda359.htm, however it states that In the above formulas ...
0
votes
0answers
4 views

What are some common sources of variance in a convolutional neural network DICE scores?

I know it's basically impossible to pinpoint the exact answer to this question without data, code, architecture, etc, but I'm curious as to if anyone has any general ideas on why this might be ...
0
votes
0answers
56 views

Finding unaffected data in a dataset

The data set has missing values. Further examination tells that they are spread along 1.5 standard deviation from the median with distribution mean = 0 & variance = 5. How much data (in percentage)...
0
votes
1answer
41 views

Initialisation methods and variance

I'm trying to understand some weights initialisation methods by reading the article http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf . But I don't understand their notation on variance. Right ...
0
votes
1answer
21 views

How is the property in eq 15 obtained for Xavier initialization

I am new in this field so please be gentle with terminology. In the original paper; "Understanding the difficulty of training deep feedforward neural networks", I dont understand how equation 15 is ...
-1
votes
0answers
16 views

bias variance decomposition for classification problem

I can see more mathematical relationship across the internet, between MSE, bias and variance. But how to do that(mathematical intuition of bias variance) for classification problem.?