Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
62
votes
11answers
34k views

What is dimensionality reduction? What is the difference between feature selection and extraction?

From wikipedia, dimensionality reduction or dimension reduction is the process of reducing the number of random variables under consideration, and can be divided into feature selection and ...
54
votes
10answers
49k views

Machine learning - features engineering from date/time data

What are the common/best practices to handle time data for machine learning application? For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
44
votes
6answers
73k views

Does scikit-learn have forward selection/stepwise regression algorithm?

I'm working on the problem with too many features and training my models takes way too long. I implemented forward selection algorithm to choose features. However, I was wondering does scikit-learn ...
30
votes
4answers
20k views

Does XGBoost handle multicollinearity by itself?

I'm currently using XGBoost on a data-set with 21 features (selected from list of some 150 features), then one-hot coded them to obtain ~98 features. A few of these 98 features are somewhat redundant, ...
29
votes
6answers
9k views

Are there any tools for feature engineering?

Specifically what I am looking for are tools with some functionality, which is specific to feature engineering. I would like to be able to easily smooth, visualize, fill gaps, etc. Something similar ...
19
votes
2answers
13k views

Text categorization: combining different kind of features

The problem I am tackling is categorizing short texts into multiple classes. My current approach is to use tf-idf weighted term frequencies and learn a simple linear classifier (logistic regression). ...
19
votes
3answers
11k views

How to perform feature engineering on unknown features?

I am participating on a kaggle competition. The dataset has around 100 features and all are unknown (in terms of what actually they represent). Basically they are just numbers. People are performing ...
17
votes
4answers
17k views

Any “rules of thumb” on number of features versus number of instances? (small data sets)

I am wondering, if there are any heuristics on number of features versus number of observations. Obviously, if a number of features is equal to the number of observations, the model will overfit. By ...
17
votes
2answers
5k views

How to choose the features for a neural network?

I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way would be to ...
17
votes
3answers
4k views

How to combine categorical and continuous input features for neural network training

Suppose we have two kinds of input features, categorical and continuous. The categorical data may be represented as one-hot code A, while the continuous data is just a vector B in N-dimension space. ...
16
votes
5answers
23k views

Feature selection vs Feature extraction. Which to use when?

Feature extraction and feature selection essentially reduce the dimensionality of the data, but feature extraction also makes the data more separable, if I am right. Which technique would be ...
15
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
15
votes
4answers
4k views

How to specify important attributes?

Assume a set of loosely structured data (e.g. Web tables/Linked Open Data), composed of many data sources. There is no common schema followed by the data and each source can use synonym attributes to ...
13
votes
1answer
7k views

What is difference between one hot encoding and leave one out encoding?

I am reading a presentation and it recommends not using leave one out encoding, but it is okay with one hot encoding. I thought they both were the same. Can anyone describe what the differences ...
13
votes
2answers
4k views

What features are generally used from Parse trees in classification process in NLP?

I am exploring different types of parse tree structures. The two widely known parse tree structures are a) Constituency based parse tree and b) Dependency based parse tree structures. I am able to ...
12
votes
3answers
11k views

Is feature selection necessary?

I would like to run some machine learning model like random forest, gradient boosting, or SVM on my dataset. There are more than 200 predictor variables in my dataset and my target classes are a ...
12
votes
1answer
16k views

Feature selection using feature importances in random forests with scikit-learn

I have plotted the feature importances in random forests with scikit-learn. In order to improve the prediction using random forests, how can I use the plot information to remove features? I.e. how to ...
12
votes
1answer
4k views

Feature importance with high-cardinality categorical features for regression (numerical depdendent variable)

I was trying to use feature importances from Random Forests to perform some empirical feature selection for a regression problem where all the features are categorical and a lot of them have many ...
11
votes
5answers
7k views

When to remove correlated variables

Can somebody please suggest what is the correct stage to remove correlated variables before feature engineering or after feature engineering ?
11
votes
4answers
4k views

Feature Extraction Technique - Summarizing a Sequence of Data

I often am building a model (classification or regression) where I have some predictor variables that are sequences and I have been trying to find technique recommendations for summarizing them in the ...
11
votes
4answers
4k views

Which one first: algorithm benchmarking, feature selection, parameter tuning?

When trying to do e.g. a classification, my approach currently is to try out various algorithm first and benchmark them perform feature selection on the best algorithm from 1 above tune the ...
11
votes
2answers
3k views

What to do when testing data has less features than training data?

Let's say we are predicting the sales of a shop and my training data has two sets of features: One about the store sales with the dates (the field "Store" is not unique) One about the store types (...
10
votes
2answers
1k views

Linear Regression and scaling of data

The following plot shows coefficients obtained with linear regression (with mpg as the target variable and all others as predictors). For mtcars dataset (here and ...
10
votes
3answers
15k views

Can GPS coordinates (latitude and longitude) be used as features in a linear model?

I have data sets that contain, among many features, GPS coordinates (latitude and longitude). I'd like to use these data sets to explore problems such as: (1) computing ETA to drive between start and ...
10
votes
4answers
9k views

Feature selection and classification accuracy relation

One of the methodology to select a subset of your available features for your classifier is to rank them according to a criterion (such as information gain) and then calculate the accuracy using your ...
10
votes
7answers
2k views

Data science projects explained step by step?

I am looking for a website or book where several practical examples are given step by step, explaining how they choose the relevant features, the model selection procedure, etc...
10
votes
4answers
2k views

How to compare the performance of feature selection methods?

There are several feature selection / variable selection approaches (see for example Guyon & Elisseeff, 2003; Liu et al., 2010): filter methods (e.g., correlation-based, entropy-based, random ...
9
votes
2answers
14k views

List of feature engineering techniques

Is there any resource with a list of feature engineering techniques? A mapping of type of data, model and feature engineering technique would be a gold mine
9
votes
1answer
391 views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
9
votes
1answer
34k views

How to do stepwise regression using sklearn? [duplicate]

I could not find a way to stepwise regression in scikit learn. I have checked all other posts on Stack Exchange on this topic. Answers to all of them suggests using f_regression. But f_regression ...
9
votes
1answer
4k views

feature importance via random forest and linear regression are different

Applied Lasso to rank the features and got the following results: ...
9
votes
1answer
890 views

Feature selection for Support Vector Machines

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
9
votes
1answer
198 views

Learning signal encoding

I have a large number of samples which represent Manchester encoded bit streams as audio signals. The frequency at which they are encoded is the primary frequency component when it is high, and there ...
9
votes
2answers
97 views

Features reduction for the not correlated data set

I am working with classification problem on a training data set, which have 100 features. All the features in pairs haven't visible correlation. One can see it in the example pair plot for the some of ...
8
votes
3answers
152 views

Feature selection for tracking user activity within an application

I am developing a system that is intended to capture the "context" of user activity within an application; it is a framework that web applications can use to tag user activity based on requests made ...
8
votes
1answer
1k views

Document classification: tf-idf prior to or after feature filtering?

I have a document classification project where I am getting site content and then assigning one of numerous labels to the website according to content. I found out that tf-idf could be very useful ...
8
votes
3answers
13k views

Improving accuracy of Text Classification

I am working on a text classification problem, the objective is to classify news articles to their corresponding categories, but in this case the categories are not very broad like, politics, sports, ...
8
votes
4answers
129 views

How to handle features which are not always available?

I have a feature in my feature vector that is not always available respectively sometimes (for some samples) it makes no sense to use it. I feed a sklearn MLPClassifier with this feature vector. Does ...
7
votes
1answer
837 views

Interpreting the results of randomized PCA in scikit-learn

I'm using scikit-learn to do a genome-wide association study with a feature vector of about 100K SNPs. My goal is to tell the biologists which SNPs are "interesting". RandomizedPCA really improved ...
7
votes
1answer
2k views

Named entity recognition (NER) features

I'm new to Named Entity Recognition and I'm having some trouble understanding what/how features are used for this task. Some papers I've read so far mention features used, but don't really explain ...
7
votes
2answers
3k views

LSTM Feature selection process

We need to implement a time series problem with the LSTM model. But, while implementing the same, the main challenge I am facing is the feature selection issue. Because our data-set contains 2300 ...
6
votes
2answers
9k views

Dissmissing features based on correlation with target variable

Is it valid to dismiss features based on their Pearson correlation values with the target variable in a classification problem? say for instance I have a dataset with the following format where the ...
6
votes
2answers
1k views

Improve a regression model and feature selection

I am working on Azure ML Studio and try to create a regression model to predict a numerical value. I will try to describe my features and what I have done until now. My data with about 3 million rows ...
6
votes
1answer
26k views

Number of features of the model must match the input. Model n_features is `N` and input n_features is `X`.

I am new to data science and trying get some results. I'm applying Decision Tree Classifier. When my train and test datasets' size are not equal I get an error `...
6
votes
2answers
3k views

Is there a model-agnostic way to determine feature importance?

Sklearn has a feature_importances_ attribute, but this is highly model-specific and I'm not sure how to interpret it as removing the most important feature does not ...
6
votes
3answers
1k views

Regression vs Random Forest - Combination of features

I had a discussion with a friend and we were talking about the advantages of random forest over linear regression. At some point, my friend said that one of the advantages of the random forest over ...
6
votes
1answer
14k views

What is the meaning of hand crafted features in computer vision problems?

Are these the features which are manually labelled by humans? or Is there any technique for obtaining these features. Is this related to learned features?
6
votes
2answers
3k views

Which feature selection I should trust?

I use Python and Weka to run feature selection on my dataset (91 predictor variables). I can see a huge difference (feature ranking) from different algorithms. And these results are still quite ...
6
votes
1answer
99 views

Will unnecessary features harm the tree based model?

Is it necessary to drop noisy features (eg column of random numbers) from tree features? I think it's not. sometimes it may benefit but will never cause any harm to the model. Because at each split ...
6
votes
1answer
1k views

Instead of one-hot encoding a categorical variable, could I profile the data and use the percentile value from it's cumulative density distribution?

I have a categorical variable which has thousands of values, for a dataset which has millions of records. The data is being used to create a binary classification model. I am in the early steps of ...

1
2 3 4 5
13