Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
278
questions with no upvoted or accepted answers
9
votes
4
answers
5k
views
How to perform feature selection on dataset with categorical and numerical features?
I am working on a dataset with 30 columns (29 numerical, 1 non-ordinal categorical). I hot-encoded the categorical feature and reached at 35 columns. To improve training efficiency, I want to perform ...
- 91
6
votes
2
answers
2k
views
Metrics to evaluate features' importance in classification problem (with random forest)
I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest.
The most widely used ones apparrently are:
Cross Entropy-Information ...
- 1,067
4
votes
0
answers
86
views
Non-Gaussian like distributions - Classifier of source data fails on target data
I ask you for help on a classification problem (classes are represented by the numbers 0,1 and 2). All features are extracted from time series data (fundamental is sinus shape).
I have a source ...
- 51
4
votes
1
answer
1k
views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
3
votes
1
answer
262
views
stacking features vs concatenating layers
I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
- 342
3
votes
0
answers
117
views
Features selection with a lot of dummy variables in R
I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
- 131
3
votes
0
answers
147
views
Feature Importance Python
My dataset has around 1000 features and 30k rows.
All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
- 243
3
votes
1
answer
315
views
Feature selection for time series prediction
I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables.
When calculating correlation between variables (e.g. Close price of Tesla vs ...
3
votes
1
answer
112
views
Determine the most important documents for supervised learning
I have somewhat of a general/high level question.
Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
- 31
3
votes
1
answer
82
views
Software for reweighted L1 minmization?
I am trying to solve a sparsity-promoting optimization problem. It is well known that the L1 norm is a good surrogate to the L0 norm, and it is studied in (Candes et al, 2008: Enhancing sparsity by ...
- 31
3
votes
1
answer
145
views
scikit-learn OMP mem error
I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error.
The machine has ...
- 31
3
votes
1
answer
909
views
XGBoost - feature importance just depends on the location of the feature in the data
I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
- 133
2
votes
0
answers
29
views
How to properly select features for time series ML models
I've been trying to get good references on how to solve a problem that's been bothering me regarding the modelling techniques I've used. I'm currently interested in making forecasts using ML for ...
- 21
2
votes
0
answers
81
views
Feature selection and model performance
Featuretools provides an automated way to generate features from your data, by providing relationships within your data and applying their so-called deep feature synthesis. It generates features like ...
- 121
2
votes
0
answers
667
views
Library for Phi correlation coefficient in python?
I want to calculate correlation b/w categorical features in my data. I reviewed the literature and found phi coefficient can be used for this purpose. I found one library called phik in python enter ...
- 77
2
votes
0
answers
18
views
Machine learning cost/benefit for including priors in input vector
Is there a trade-off in accuracy/generalisation/performance when providing priors to a general machine learning algorithm vs training the machine learning algorithm with enough data so that it could ...
- 121
2
votes
0
answers
29
views
Sequential feature selection stopping condition
When using sequential feature selection approach, say forward feature selection, I want to stop adding new features when the improvement in model scores is smaller than a certain level. Is there a ...
- 21
2
votes
0
answers
26
views
Comparing machine learning algorithms on features selected by a neural network
I'm reading a paper where they use a neural network to select 9 features from tabular input data with 20 features. And then, this is what feels weird to me, they run several machine learning ...
- 61
2
votes
0
answers
49
views
Random Forest importances vs Feature Permutation: cummulative sum of importances are 1 and 0.1, respectively. Make sense?
I am performing feature selection by using two methods: MDI (RandomForest importances) and Feature Permutation, in order to compare what are the features considered relevant for both methods. My ...
- 121
2
votes
0
answers
32
views
Why are correlation matrices used versus a matrix of R^2 values?
I'm relatively new to DS, so forgive me if this is a dumb question or in the wrong forum
When evaluating features it seems that almost everywhere a correlation matrix is used ...
- 21
2
votes
2
answers
115
views
Static ML model or Time-Series? How to model/predict a binary target when I have time variant features but most features are constant?
I have been working with Real World data from patients. I have a dataset with information about 10million patients; Collected over a span of varying duration (5 to 20 years).
What I am predicting is ...
- 21
2
votes
0
answers
35
views
Using on-demand features in machine learning
I have 6 input features $[m1,m2,m3,m4,m5,m6]$.
I am trying to build a model that can predict the value of all 6 of these values using $[m1,m2,m3]$. However, I have the option of asking for another ...
- 121
2
votes
0
answers
24
views
Determine most important features in diagnostic data
I have a dataset of device diagnostics. I have two tables: one relating each device to failures code. Two devices can share a failure code for example a common chip malfunction. The second table links ...
- 21
2
votes
1
answer
226
views
How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?
I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
- 21
2
votes
0
answers
245
views
Shapley summary plot interpretation doubt?
I have question when interpreting SHAP summary plot. I have attached the sample plot
Here, If I am interpreting it correctly, low values of feature 1 are associated with high and negative values for ...
- 21
2
votes
1
answer
241
views
feature importance with CNN
I can use shap to extract important features for Dense NN. However, for CNN, I encountered two problems:
the feature order may be messed up or combined after the ...
- 21
2
votes
2
answers
3k
views
How to perform feature selection with Categorical Variables and Continuous Target, provided that data is not normally distributed?
I am trying a use multi linear regression model to predict the salaries of employees. I have a total of 88 dependent features from which 19 are categorical and the rest are continuous. I have managed ...
- 31
2
votes
1
answer
127
views
Adding high p-value and low R square features in linear regression model to improve result
I am working on a linear regression problem. The features for my analysis have been selected using p-values and domain knowledge. After selecting these features, the performance of $R^2$ and the $...
2
votes
1
answer
377
views
How to interpret a specific feature importance?
Apologies for a very case specific question. I have a dataset of genes, with which I am using machine learning to predict if a gene causes a disease. One of the features I have is a beta value (which ...
- 185
2
votes
0
answers
130
views
Model-independent measures for feature importance given highly correlated features
I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
- 131
2
votes
1
answer
73
views
features to help distinguish between document images
we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
- 342
2
votes
0
answers
133
views
How to do Multivariate Adaptive Regression Splines feature selection in python? Specifically, I need the python equivalent of the earth function in R
This is the code in R:
marsModel <- earth(eval(parse(text=paste(ResponseVariable,"~."))), data = data) #build model
ev <- evimp (marsModel)
Response ...
- 21
2
votes
1
answer
340
views
Logistic Regression Model for categorical features with multiple values in each category
I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
- 173
2
votes
1
answer
227
views
Feature Selection on Aggregated Targetdata
I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
2
votes
0
answers
136
views
When does random forest feature importance fail?
I'm curious about the assumptions of random forest feature importance.
In this paper, the author says that
"We show that random forest variable importance measures are a sensible means for ...
- 149
2
votes
0
answers
150
views
Structured Support Vector Machine (Joint Feature Map)
I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine)
The theory's clear, but I need a tangible example to make everything more concrete.
...
2
votes
0
answers
1k
views
Combine multiple features for text classification
Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I'm trying to make my own classification algorithm (the text sends a positive/...
- 75
2
votes
1
answer
38
views
Classifying objects based of a varying number of the same type of feature vector for each object
For a congressional session, I have created a doc2vec model of speeches made. Using the vectors from this model, I have a dataset of each congressperson, their political affiliation, and a list of the ...
- 145
2
votes
0
answers
102
views
Representing a community as a vector
My setup is this:
Suppose I have transactional data over a large period of time. The parties of each transaction are labled, and I use Louvain algorithm for detecting communities (and sub-communities)...
- 121
2
votes
1
answer
258
views
statistical significance test between binary label features
I have 667 features and I want to find features that have a significant boundary between a binary class label before I apply a classification model (e.g Naive Bayes/ SVM) to improve classification ...
- 33
2
votes
0
answers
220
views
Giving Emails as Input to Machine Learning Algorithms
I want to classify emails as Spam and Non-Spam.
I have a labelled dataset of 20,000 emails in TXT format. The emails are in individual files and also in one combined file.
An example email looks ...
- 121
2
votes
0
answers
2k
views
Feature extraction using autoencoder and assigning sub-features to the classes
I have a dataset with N records and D numerical attributes belonign to C different classes. ...
- 1,255
2
votes
0
answers
45
views
Am I correct in finding correlations
I want to perform feature selection, having 128 real-valued standardized features and 1/0 labels. Below are feature a5 density histograms for Classes 1 and 0. The data is skewed, so that Class 1 is ...
- 323
2
votes
1
answer
2k
views
Variance threshold with python problem
i’am a beginner in scikit-learn and i’ve a little problem when using feature selection module VarianceThreshold, the problem is when i set the variance ...
- 51
2
votes
0
answers
515
views
randomForest::varImp VS conditional variable importance
Data:
My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
- 193
2
votes
0
answers
118
views
What's the strategy for deciding which feature level is excluded from one hot encoding of a categorical variable?
I'm working on a regression problem with a continuous dependent variable (sale price of a home). Amongst my features are several categorical features, which I'm transforming to "one hot encoded" dummy ...
- 521
2
votes
0
answers
76
views
Parallel processing for feature selection in microarray dataset
I want to apply feature selection on a dataset with some 30-40K columns and 100 rows ( total size: 400MB-800MB ). To decrease the time consumed for calculations involved (feature-feature), I want to ...
- 21
2
votes
0
answers
46
views
You are in charge of investing Chipotle's E-Coli source/s, what methods do you use?
With the amount of geographic diversity that is popping up over time with Chipotle and the outbreak of e-coli, if you were investigating the source of all these issues, and where it could occur again ...
- 21
2
votes
1
answer
235
views
Underlying model for prediction using different prediction variables
I have time-series energy consumption data for a duration of one-month. The frequency of data is half-hourly. The features of dataset are
temperature - temperature value at particular time instant
...
- 300
1
vote
1
answer
36
views
how to handle a variable number of feature-values (1:many) without one-hot
I am using Catboost and one thing I notice in the guide is that it says to not preprocess to one-hot encoding.
My data has a single target per row however the feature can have both thousands of values ...
- 111