Skip to main content
added 315 characters in body
Source Link
Hack-R
  • 1.9k
  • 1
  • 21
  • 34

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

One way you use PCA for feature selection is to look at the factor loading on the principal components and determine which correlated variables are measuring the same principal component then pick the top 1 or few variables to represent that latent variable, eliminating highly correlated non-distinct features.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Some additional tools for feature selection:

  • Minimum Redundancy Maximum Relevance
  • Correlation Feature Selection
  • Canonical Correlations analysis
  • Factor Analysis
  • Use of a covariance matrix
  • Singular Value Decomposition
  • Variance Inflation Factors

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Some additional tools for feature selection:

  • Minimum Redundancy Maximum Relevance
  • Correlation Feature Selection
  • Canonical Correlations analysis
  • Factor Analysis
  • Use of a covariance matrix
  • Singular Value Decomposition
  • Variance Inflation Factors

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

One way you use PCA for feature selection is to look at the factor loading on the principal components and determine which correlated variables are measuring the same principal component then pick the top 1 or few variables to represent that latent variable, eliminating highly correlated non-distinct features.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Some additional tools for feature selection:

  • Minimum Redundancy Maximum Relevance
  • Correlation Feature Selection
  • Canonical Correlations analysis
  • Factor Analysis
  • Use of a covariance matrix
  • Singular Value Decomposition
  • Variance Inflation Factors
added 282 characters in body
Source Link
Hack-R
  • 1.9k
  • 1
  • 21
  • 34

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy?Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

Should you eliminate redundant features before PCA?Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Some additional tools for feature selection:

  • Minimum Redundancy Maximum Relevance
  • Correlation Feature Selection
  • Canonical Correlations analysis
  • Factor Analysis
  • Use of a covariance matrix
  • Singular Value Decomposition
  • Variance Inflation Factors

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.

Some additional tools for feature selection:

  • Minimum Redundancy Maximum Relevance
  • Correlation Feature Selection
  • Canonical Correlations analysis
  • Factor Analysis
  • Use of a covariance matrix
  • Singular Value Decomposition
  • Variance Inflation Factors
Source Link
Hack-R
  • 1.9k
  • 1
  • 21
  • 34

Redundant features can be features that are multicolinear (i.e. highly correlated), but more importantly they're measuring the same thing without a unique contribution.

For instance, age and income might be highly correlated, but in some analyses they still have a unique effect in your model and may have conceptual differences that you want to captured for interpretation. OTOH, age and birth date are purely redundant in most use cases I can think of (though there are always exceptions, such as if season of birth is important).

Can PCA help reduce redundancy? Sure. It's one of at least dozens of techniques you could use for this.

Should you eliminate redundant features before PCA? If you're going to use the principal components for prediction rather than feature elimination, then yes.

You can do one round of feature analysis involving PCA or other techniques and a second round to create latent variables for your model if you want to do both.