Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
284
questions with no upvoted or accepted answers
7
votes
4
answers
4k
views
How to perform feature selection on dataset with categorical and numerical features?
I am working on a dataset with 30 columns (29 numerical, 1 non-ordinal categorical). I hot-encoded the categorical feature and reached at 35 columns. To improve training efficiency, I want to perform ...
- 71
6
votes
2
answers
2k
views
Metrics to evaluate features' importance in classification problem (with random forest)
I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest.
The most widely used ones apparrently are:
Cross Entropy-Information ...
- 1,057
4
votes
1
answer
1k
views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
3
votes
0
answers
78
views
Non-Gaussian like distributions - Classifier of source data fails on target data
I ask you for help on a classification problem (classes are represented by the numbers 0,1 and 2). All features are extracted from time series data (fundamental is sinus shape).
I have a source ...
- 41
3
votes
1
answer
188
views
stacking features vs concatenating layers
I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
- 328
3
votes
0
answers
107
views
Features selection with a lot of dummy variables in R
I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
- 31
3
votes
0
answers
144
views
Feature Importance Python
My dataset has around 1000 features and 30k rows.
All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
- 243
3
votes
1
answer
289
views
Feature selection for time series prediction
I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables.
When calculating correlation between variables (e.g. Close price of Tesla vs ...
3
votes
1
answer
91
views
Determine the most important documents for supervised learning
I have somewhat of a general/high level question.
Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
- 31
2
votes
0
answers
79
views
Feature selection and model performance
Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
- 121
2
votes
0
answers
14
views
Machine learning cost/benefit for including priors in input vector
Is there a trade-off in accuracy/generalisation/performance when providing priors to a general machine learning algorithm vs training the machine learning algorithm with enough data so that it could ...
- 121
2
votes
0
answers
27
views
Sequential feature selection stopping condition
When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
- 21
2
votes
1
answer
80
views
Using cross validation score to perform feature selection
So to perform my feature selection I ran cross validation over and over again, each time trying different subsets of my attributes and repeated this until I got the best cross validation score I could ...
- 21
2
votes
0
answers
37
views
Random Forest importances vs Feature Permutation: cummulative sum of importances are 1 and 0.1, respectively. Make sense?
I am performing feature selection by using two methods: MDI (RandomForest importances) and Feature Permutation, in order to compare what are the features considered relevant for both methods. My ...
- 121
2
votes
0
answers
28
views
Why are correlation matrices used versus a matrix of R^2 values?
I'm relatively new to DS, so forgive me if this is a dumb question or in the wrong forum
When evaluating features it seems that almost everywhere a correlation matrix is used ...
- 21
2
votes
1
answer
56
views
Static ML model or Time-Series? How to model/predict a binary target when I have time variant features but most features are constant?
I have been working with Real World data from patients. I have a dataset with information about 10million patients; Collected over a span of varying duration (5 to 20 years).
What I am predicting is ...
- 21
2
votes
0
answers
28
views
Using on-demand features in machine learning
I have 6 input features $[m1,m2,m3,m4,m5,m6]$.
I am trying to build a model that can predict the value of all 6 of these values using $[m1,m2,m3]$. However, I have the option of asking for another ...
- 121
2
votes
0
answers
21
views
Determine most important features in diagnostic data
I have a dataset of device diagnostics. I have two tables: one relating each device to failures code. Two devices can share a failure code for example a common chip malfunction. The second table links ...
- 21
2
votes
1
answer
186
views
How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?
I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
- 21
2
votes
0
answers
221
views
Shapley summary plot interpretation doubt?
I have question when interpreting SHAP summary plot. I have attached the sample plot
Here, If I am interpreting it correctly, low values of feature 1 are associated with high and negative values for ...
- 21
2
votes
1
answer
157
views
feature importance with CNN
I can use shap to extract important features for Dense NN. However, for CNN, I encountered two problems:
the feature order may be messed up or combined after the ...
- 21
2
votes
1
answer
382
views
Feature Selection and Outlier Detection
How does feature selection impact outlier detection and also, removing outliers impact feature selection?
It could be a basic question. However, just to know the boundaries, I asked. Thanks in advance....
- 161
2
votes
2
answers
2k
views
How to perform feature selection with Categorical Variables and Continuous Target, provided that data is not normally distributed?
I am trying a use multi linear regression model to predict the salaries of employees. I have a total of 88 dependent features from which 19 are categorical and the rest are continuous. I have managed ...
- 31
2
votes
1
answer
119
views
Adding high p-value and low R square features in linear regression model to improve result
I am working on a linear regression problem. The features for my analysis have been selected using p-values and domain knowledge. After selecting these features, the performance of $R^2$ and the $...
2
votes
1
answer
343
views
How to interpret a specific feature importance?
Apologies for a very case specific question. I have a dataset of genes, with which I am using machine learning to predict if a gene causes a disease. One of the features I have is a beta value (which ...
- 155
2
votes
0
answers
126
views
Model-independent measures for feature importance given highly correlated features
I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
- 131
2
votes
1
answer
58
views
features to help distinguish between document images
we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
- 328
2
votes
0
answers
123
views
How to do Multivariate Adaptive Regression Splines feature selection in python? Specifically, I need the python equivalent of the earth function in R
This is the code in R:
marsModel <- earth(eval(parse(text=paste(ResponseVariable,"~."))), data = data) #build model
ev <- evimp (marsModel)
Response ...
- 21
2
votes
1
answer
259
views
Logistic Regression Model for categorical features with multiple values in each category
I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
- 173
2
votes
1
answer
190
views
Feature Selection on Aggregated Targetdata
I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
2
votes
0
answers
132
views
When does random forest feature importance fail?
I'm curious about the assumptions of random forest feature importance.
In this paper, the author says that
"We show that random forest variable importance measures are a sensible means for ...
- 139
2
votes
1
answer
924
views
Hierarchical Clustering and Variable Selection
I am using "Single linkage" hierarchical algorithm to cluster my data points with Gower Distance as my data have both qualitative and quantitative variables.
After applying this for the full ...
2
votes
0
answers
144
views
Structured Support Vector Machine (Joint Feature Map)
I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine)
The theory's clear, but I need a tangible example to make everything more concrete.
...
2
votes
0
answers
1k
views
Combine multiple features for text classification
Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I'm trying to make my own classification algorithm (the text sends a positive/...
- 75
2
votes
1
answer
37
views
Classifying objects based of a varying number of the same type of feature vector for each object
For a congressional session, I have created a doc2vec model of speeches made. Using the vectors from this model, I have a dataset of each congressperson, their political affiliation, and a list of the ...
- 145
2
votes
0
answers
101
views
Representing a community as a vector
My setup is this:
Suppose I have transactional data over a large period of time. The parties of each transaction are labled, and I use Louvain algorithm for detecting communities (and sub-communities)...
- 121
2
votes
0
answers
214
views
Giving Emails as Input to Machine Learning Algorithms
I want to classify emails as Spam and Non-Spam.
I have a labelled dataset of 20,000 emails in TXT format. The emails are in individual files and also in one combined file.
An example email looks ...
- 121
2
votes
0
answers
2k
views
Feature extraction using autoencoder and assigning sub-features to the classes
I have a dataset with N records and D numerical attributes belonign to C different classes. ...
- 1,225
2
votes
0
answers
44
views
Am I correct in finding correlations
I want to perform feature selection, having 128 real-valued standardized features and 1/0 labels. Below are feature a5 density histograms for Classes 1 and 0. The data is skewed, so that Class 1 is ...
- 323
2
votes
1
answer
2k
views
Variance threshold with python problem
i’am a beginner in scikit-learn and i’ve a little problem when using feature selection module VarianceThreshold, the problem is when i set the variance ...
- 51
2
votes
0
answers
508
views
randomForest::varImp VS conditional variable importance
Data:
My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
- 193
2
votes
0
answers
117
views
What's the strategy for deciding which feature level is excluded from one hot encoding of a categorical variable?
I'm working on a regression problem with a continuous dependent variable (sale price of a home). Amongst my features are several categorical features, which I'm transforming to "one hot encoded" dummy ...
- 521
2
votes
1
answer
74
views
Software for reweighted L1 minmization?
I am trying to solve a sparsity-promoting optimization problem. It is well known that the L1 norm is a good surrogate to the L0 norm, and it is studied in (Candes et al, 2008: Enhancing sparsity by ...
- 21
2
votes
1
answer
135
views
scikit-learn OMP mem error
I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error.
The machine has ...
- 21
2
votes
1
answer
808
views
XGBoost - feature importance just depends on the location of the feature in the data
I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
- 123
2
votes
1
answer
232
views
Underlying model for prediction using different prediction variables
I have time-series energy consumption data for a duration of one-month. The frequency of data is half-hourly. The features of dataset are
temperature - temperature value at particular time instant
...
- 300
1
vote
0
answers
16
views
1
vote
2
answers
55
views
Feature selection - How to identify the best subset
I am using three feature selection method on a dataset containing 15 inputs. I need to extract the best 5 features. Each of the three method gave a subset of the input dataset, but they are different. ...
- 11
1
vote
0
answers
20
views
How to implement Ant Lion Optimization (ALO) feature selection for KNN Classification Problems?
I have been assigned for a project related to text data classification, i have preprocess and vectorized the data with TF-IDF. For feature selection i am using pyMetahueristic library to implement ALO ...
- 11
1
vote
0
answers
29
views
How recursive feature elimination with cross validation internally works?
I am trying to understand how recursive feature elimination with cross validation works (the RFECV on sklearn). Lets say that we have 10 features, and we perform <...
- 181