1
$\begingroup$

Analysing 3 features I was faced with the following situation: Two of them have almost no variance and the last one have a greater variance than the other (almost two times). Is it correct to assume using only this information that the feature with more variance will be more important to my model ? Also, if a feature has 0 variance the model simply won't learn anything from it, am I right?

Lastly, if you have some nice references I would love to read them.

$\endgroup$
2
  • $\begingroup$ High variance doesn't necessarily mean that the feature is more important. What type of model are you talking about? $\endgroup$ – bstrain Nov 6 '19 at 3:42
  • $\begingroup$ Linear regression $\endgroup$ – nzBoan Nov 7 '19 at 9:10
2
$\begingroup$

High feature variance does not imply any sort of relationship to the target variable you're interested in modeling. Suppose you are looking at a population and have two variables for each person, their height in meters and annual income in dollars. The variance of height will be a small number (heights don't vary by much more than 1 meter), but the variance of income will be a large number (incomes can vary by thousands). If you want to predict the person's arm span, their height will be a much better predictor than their income, despite the fact that it has a numerically smaller variance.

A zero variance feature does not have any predictive power, but even numerically small fluctuations can be very important depending on your goal.

$\endgroup$
0
$\begingroup$

A feature will zero variance will have no predictive ability in a model. Zero variance means the feature values are constant across different target values.

Features with higher variance have the potential for more predictive ability. However, it depends on the specific problem. The predictive ability of features can only be known after fitting a model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.