Questions tagged [variance]
The variance tag has no usage guidance.
1
vote
1answer
18 views
Bias-variance tradeoff in practice (CNN)
I first trained a CNN on my dataset and got a loss plot that looks somewhat like this:
Orange is training loss, blue is dev loss. As you can see, the training loss is lower than the dev loss, so I ...
0
votes
0answers
8 views
Feature selection: variance vs. Index of Dispersion
I want to remove some features of my dataset before the training process. I'd like to use the VarianceThreshold method from Sklearn. However, this requires the data to be in the same range 0-1. In ...
0
votes
1answer
31 views
Linear machine learning algorithms “often” have high bias/low variance?
In this blog, which explains the meaning of bias and variance in machine learning, there's a line under the heading "Bias-Variance Trade-Off"
which says:
Parametric or linear machine learning ...
0
votes
4answers
61 views
Meaning of variance in machine learning models
I know that high variance cause overfitting, and high variance is that the model is sensitive to outliers.
But can I say Variance is that when the predicted points are too prolonged lead to high ...
0
votes
1answer
38 views
Initialisation methods and variance
I'm trying to understand some weights initialisation methods by reading the article http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf .
But I don't understand their notation on variance. Right ...
0
votes
0answers
11 views
Glorot Initialisation: Why is it important for the input and output variance to be the same?
In this blog post about Glorot initialisation, the derivation proceeds by calculating the variance of the output $Y$ of a linear neuron. At one point in the derivation it is shown that $Var(Y) = nVar(...
0
votes
0answers
7 views
Measuring time series variability with polarity changes
0
down vote
favorite
I am trying to measure the variability or "choppiness" of a time series but I am aware that standard measures of standard deviation do not apply do to auto-correlations between ...
0
votes
0answers
10 views
Predicting relative performance of algorithm configurations on a task
Setup
I have a set of tasks $T = t_0, \dots, t_k$, and an algorithm that can be configured with a set of settings $S = s_0, \dots, s_m$. Each task $t_i$ also has a set of properties $P_i = p_0, \dots,...
0
votes
2answers
69 views
Relationship between train and test error
I have some specific questions for which I could not extract answers from books. Therefore, I ask for help here and shall be extremely grateful for an intuitive explanation if possible.
In general, ...
2
votes
1answer
582 views
RL Advantage function why A = Q-V instead of A=V-Q?
In RL Course by David Silver - Lecture 7: Policy Gradient Methods, David explains what an Advantage function is, and how it's the difference between Q(s,a) and the V(s)
Preliminary, from this post:
...
2
votes
3answers
79 views
What is the meaning of term Variance in Machine Learning Model?
I am familiar with terms high bias and high variance and their effect on the model.
Basically your model has high variance when it is too complex and sensitive too even outliers.
But recently I was ...
1
vote
0answers
68 views
Evaluation of regression models with different evaluations (MSE, variance, VAF etc.)
When comparing several regression models in terms of quality, it seems like most have agreed on the MSE.
There are also papers comparing "variance" and "variance accounted for (VAF)".
However, there ...
0
votes
1answer
60 views
Maths question on mean squared error being dervied to bias and variance
I am reading a book and have difficulty in understanding the math on bias-
variance tradeoff. Below is the section that I am having trouble with:
Given a set of training samples $x_1, x_2, ..., x_n$...
1
vote
1answer
22 views
sort occurrence matrix to minimize its spatial variance
Is there any algorithm or an approach that sort occurrence matrix to reduce its spatial variance. I mean by spatial variance moving window variance (n*n moving box).
0
votes
0answers
35 views
Is it possible to compute residual variances of Laplacian Eigenmaps, Diffusion Map and t-SNE to have a scree plots as in PCA?
I will like to do comparative analysis of PCA vs. Laplacian Eigenmaps, Diffusion Map, and <...
0
votes
0answers
160 views
How to understand the bias term b and how it is updated in ADALINE algorithm?
With $y=x*w+b$, where $x$ is the feature vector of a sample and $w$ is the weight vector, the update rule (SGD) for the bias $b$ is: $b \leftarrow b + \eta(o - y)$.
With Gradient Descent, the final $...
1
vote
2answers
179 views
How is the equation for the relation between prediction error, bias, and variance defined?
I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned:
If we denote the variable we are trying to predict as $Y$ and our
covariates as $X$, we may assume that there is a ...
0
votes
1answer
20 views
How is the property in eq 15 obtained for Xavier initialization
I am new in this field so please be gentle with terminology.
In the original paper; "Understanding the difficulty of training deep feedforward neural networks", I dont understand how equation 15 is ...
4
votes
2answers
390 views
How to estimate the variance of regressors in scikit-learn?
Every classifier in scikit-learn has a method predict_proba(x) that predicts class probabilities for x. How to do the same thing ...
1
vote
0answers
254 views
How to sort tensor along two axis by variance in TensorFlow?
I have a function that visualizes 2D convolutional kernels. The weights come as 4D-tensors [filter_height, filter_width, in_channels, out_channels], as specified for conv2d. Now I try to sort the ...
1
vote
1answer
727 views
How to decide what threshold to use for removing low-variance features?
How to decide what threshold to use for removing low-variance features?
Particularly, I have 100000 features and the variances look like:
Could I e.g. take the average and use it to split this to ~...
4
votes
2answers
64 views
Trade off between Bias and Variance [closed]
What are the best ideas or approaches to trade off between bias and variance in Machine Learning models.
8
votes
1answer
194 views
Question on bias-variance tradeoff and means of optimization
So I was wondering how does one, for example, can best optimize the model they are trying to build when confronted with issues presented by high bias or high variance. Now, of course, you can play ...
0
votes
1answer
47 views
very low variance explained after applying pca
I applied PCA on MNIST data and found that the first 64 components are able to retain 86% of variance.
Is there any problem while applying pca to a big dataset like MNIST. Because in most of the ...
1
vote
1answer
70 views
Convert a pdf into a conditional pdf such that mean increases and std dev falls
Let success metric(for some business use case I am working on) be a continuous random variable S.
The mean of pdf defined on S indicates the chance of success. Higher the mean more is the chance of ...
3
votes
1answer
38 views
Bootstrapping or Randomly Dividing Dataset to reduce variance?
If I have 10,000 training samples then what should I do:
Bootstrapping and train 10 classifiers on it and then aggregating
Or
randomly divide the dataset into 10 parts and train 10 classifiers on ...
1
vote
2answers
75 views
How can I calculate mean and variance incrementally?
Say I have a set S of values, and want to store in a database some summary information about that set, so that later when I acquire a new value v I can make a reasonable estimate of what the summary ...
4
votes
2answers
568 views
How do you set sigma for the Gaussian similarity kernel?
Let's say we have $n$ two-dimensional vectors: $$\mathbf{x}_1,\dots,\mathbf{x}_i,\dots,\mathbf{x}_n=(x_{1_1},x_{1_2})^T,\dots,(x_{i_1},x_{i_2})^T,\dots,(x_{n_1},x_{n_2})^T$$ How do you set $\sigma$ ...
7
votes
3answers
2k views
Overfitting Naive Bayes
My question is what are potential reasons for Naive Bayes to perform well on a train set but poorly on a test set?
I am working with a variation of the 20news dataset. The dataset has documents, ...
0
votes
1answer
339 views
How to measure variance in a classification dataset?
I have a dataset that contains 20 predictor variables (both categorical and numeric) and one target variable (2 classes - Good and Bad). But, there are only 23 observations in the dataset. While I ...
