A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
48
votes
10answers
41k views

Machine learning - features engineering from date/time data

What are the common/best practices to handle time data for machine learning application? For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
60
votes
11answers
32k views

What is dimensionality reduction? What is the difference between feature selection and extraction?

From wikipedia, dimensionality reduction or dimension reduction is the process of reducing the number of random variables under consideration, and can be divided into feature selection and ...
38
votes
5answers
63k views

Does scikit-learn have forward selection/stepwise regression algorithm?

I'm working on the problem with too many features and training my models takes way too long. I implemented forward selection algorithm to choose features. However, I was wondering does scikit-learn ...
24
votes
4answers
16k views

Does XGBoost handle multicollinearity by itself?

I'm currently using XGBoost on a data-set with 21 features (selected from list of some 150 features), then one-hot coded them to obtain ~98 features. A few of these 98 features are somewhat redundant, ...
4
votes
1answer
2k views

Cleaning input data with pd.get_dummies()

What is the advantage of converting a series like >>> df Color 0 Red 1 Blue 2 Green 3 Red To a multiple series like the below? ...
14
votes
5answers
19k views

Feature selection vs Feature extraction. Which to use when?

Feature extraction and feature selection essentially reduce the dimensionality of the data, but feature extraction also makes the data more separable, if I am right. Which technique would be ...
14
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
10
votes
3answers
10k views

Is feature selection necessary?

I would like to run some machine learning model like random forest, gradient boosting, or SVM on my dataset. There are more than 200 predictor variables in my dataset and my target classes are a ...
4
votes
3answers
2k views

Categorizing Customer Emails

I am working on a project for a company which needs to categorize customer e-mails regarding loans and insurance. The e-mails are labeled uniquely from set of 13 category labels. The number of records ...
9
votes
3answers
13k views

Can GPS coordinates (latitude and longitude) be used as features in a linear model?

I have data sets that contain, among many features, GPS coordinates (latitude and longitude). I'd like to use these data sets to explore problems such as: (1) computing ETA to drive between start and ...
2
votes
1answer
6k views

How to determine feature importance in a neural network?

I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
1
vote
2answers
78 views

Regression Algorithms in Production

I am interested in predicting if a doctor would prescribe a specific drug and have chosen Logistic Regression as a starting point. I have a few questions: Is feature selection the first step to take ...
11
votes
2answers
851 views

Linear Regression and scaling of data

The following plot shows coefficients obtained with linear regression (with mpg as the target variable and all others as predictors). For mtcars dataset (here and ...
29
votes
6answers
8k views

Are there any tools for feature engineering?

Specifically what I am looking for are tools with some functionality, which is specific to feature engineering. I would like to be able to easily smooth, visualize, fill gaps, etc. Something similar ...
12
votes
1answer
3k views

Feature importance with high-cardinality categorical features for regression (numerical depdendent variable)

I was trying to use feature importances from Random Forests to perform some empirical feature selection for a regression problem where all the features are categorical and a lot of them have many ...
12
votes
1answer
15k views

Feature selection using feature importances in random forests with scikit-learn

I have plotted the feature importances in random forests with scikit-learn. In order to improve the prediction using random forests, how can I use the plot information to remove features? I.e. how to ...
12
votes
1answer
6k views

What is difference between one hot encoding and leave one out encoding?

I am reading a presentation and it recommends not using leave one out encoding, but it is okay with one hot encoding. I thought they both were the same. Can anyone describe what the differences ...
5
votes
2answers
8k views

Dissmissing features based on correlation with target variable

Is it valid to dismiss features based on their Pearson correlation values with the target variable in a classification problem? say for instance I have a dataset with the following format where the ...
3
votes
1answer
2k views

Is there any difference between feature extraction and feature learning?

It appears to me that "feature extraction" and "feature learning" are equivalent concepts, however there are 2 separate wikipedia articles dedicated to them that are notably different. In particular, ...
3
votes
1answer
529 views

How to visualize data of a multidimensional dataset (TIMIT)

I've built a neural network for a speech recognition task using the timit dataset. I've extracted features using the perceptual linear prediction (PLP_ method. My features space has 39 dimensions (13 ...
3
votes
2answers
53 views

Should features be correlated on uncorrelated for classification and regression (prediction)

I have seen researchers using pearson's correlation coefficient to find out the relevant features -- to keep the features that have a high correlation value with the target. The physical implication ...
2
votes
2answers
88 views

How to select features when performing classification with a dataframe of multiple columns?

I have a dataframe of 50000 observations and I want to perform a classification task. But I'm struggling with features selection. I have 89 columns, which after getting rid of some redundant features, ...
2
votes
1answer
2k views

What measures can I use to find correlation between categorical features and binary label?

For analyzing numerical features, we have correlation. What measures do we have to analyse the relevance of a categorical feature to the target value? If there isn't a direct measure, how can we ...
2
votes
3answers
445 views

Feature Selection and PCA

I have a classification problem. I want to reduce number of features to 4 (I have 30). I'm wondering why I get better result in classification when I use correlation based feature selection(cfs) first ...
8
votes
4answers
123 views

How to handle features which are not always available?

I have a feature in my feature vector that is not always available respectively sometimes (for some samples) it makes no sense to use it. I feed a sklearn MLPClassifier with this feature vector. Does ...
3
votes
1answer
83 views

Optimizing Expensive Functions

I'm trying some different techniques to optimise a Boosted Gradient Regressor by using an evolutionary programming technique to try and find the most efficient set of features. So far I've been having ...
2
votes
1answer
116 views

Different runs of feature selection algorithm giving different set of selected feature. How to choose the best set among them?

I am using the forward feature selection algorithm from MATLAB. The code is as follows: ...
2
votes
1answer
160 views

Feature selection where adding features are deteriorating model

k I am training a kNN classifier with 144 features and graphed the accuracy vs number of features used and got this. What might be the reason for the drops in the accuracy at some points of the graph?...
1
vote
1answer
48 views

How to interpret shapley force plot for feature importance?

I am trying to practice and learn shapley value approach to explain my predictions on a binary classification problem. However am having difficulty in understanding the below plot. 1) Does it ...
1
vote
3answers
5k views

is it possible to do feature selection for unsupervised machine learning problems?

I started looking for ways to do feature selection in machine learning. By having a quick look at this post , I made the assumption that feature selection is only manageable for supervised learning ...
0
votes
1answer
40 views

How to interpret Variance Inflation Factor (VIF) results?

From various books and blog posts, I understood that the Variance Inflation Factor (VIF) is used to calculate collinearity. They say that VIF till 10 is good. But I have a question. As we can see in ...