Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
572
questions
1
vote
1answer
23 views
Correlation based Feature Selection vs Feature Engineering
I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa.
Let's say I just want to get the best possible performance on a couple of models like a neural ...
1
vote
1answer
30 views
How to get feature importance from a keras deep learning model?
In case of scikit-learn's models, we can get feature importance using the relevant attributes of the model.
I've been working on a RNN, using LSTMs for text embedding.
Is there any way to get ...
1
vote
0answers
13 views
Model-independent measures for feature importance given highly correlated features
I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
0
votes
1answer
27 views
What is the name of this statistical interaction?
What is the name of the following statistical / informational interaction:
given A, I know exactly what B is.
given B, I know to some extent what A is.
I'm not looking for a probability but rather ...
1
vote
0answers
13 views
Do you need to perform variables reduction for tree-based models?
I know for methods and linear regression, GLM, Logistic regression, we typically run through a lot of variable reduction methods, i.e, forward/backward/stepwise, univariate analysis; variable ...
5
votes
3answers
453 views
What can be done with highly correlated variables (>.95 and <-.95)
I hope we can remove the highly correlated variables based on the feature importance may be with PCA etc.
Is there anything we can do with highly correlated variables/
Thanks in advance !
-3
votes
1answer
27 views
How to do feature selection for classification problem? Which technique will work? [closed]
I have 200 variables with 200000 records. How to find best features from this variables? I have tried correlation technique via Heatmap but all the variables have near to same correlation score < 0....
1
vote
0answers
16 views
How do decision tree works for feature selection?
I have a dataframe with a feature selection problem. I want to get the variables explaining the variance within each segment of the following dataset:
...
0
votes
0answers
8 views
Sequential Forward Selection (SFS) for standard Feed Forward Neural Network
I'm comparing the classification performance (accuracy, f1-score etc.) of several predictive models (logistic regression, random forest, xgboost etc.) with a standard feedforward neural network. For ...
1
vote
1answer
21 views
How to use the $\chi^2$ test to select the features, that can be String or categorical?
I want statistics to select the characteristics that have the greatest relationship to the output variable.
Thanks to this article, I learned that the scikit-learn ...
1
vote
0answers
21 views
How can we assess the importance of the features even if we ended up applying PCA?
There are multiple techniques to analyze the feature importance (permutations, SHAP values, etc).
It is essential that, in order to improve the interpretability of the model, we can somehow map the ...
2
votes
0answers
12 views
stacking features vs concatenating layers
I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
2
votes
0answers
11 views
features to help distinguish between document images
we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
2
votes
1answer
24 views
Several independent variables based on the same underlying data
I got a data containing, among others, two feature variables, which are based from the same underlying data (i.e. have mutual information), but they convey different information/message. How to handle ...
1
vote
0answers
11 views
How to do Multivariate Adaptive Regression Splines feature selection in python? Specifically, I need the python equivalent of the earth function in R
This is the code in R:
marsModel <- earth(eval(parse(text=paste(ResponseVariable,"~."))), data = data) #build model
ev <- evimp (marsModel)
Response ...
0
votes
1answer
43 views
Feature Importance from a GridSearchCV
I created a GridSearchCV for a Random Forest Regressor. Now i want to check the feature importance. I searched around and I found this
...
3
votes
1answer
45 views
Should Feature Selection processes be apply on training data or on all data?
I've realized that on examples and guides, sometimes feature selection processes (correlation elimination, backward/stepwise) are applied on the train data after splitting all data but on the other ...
1
vote
0answers
39 views
How to predict using the pickle model?
I am new to data science and trying to learn something. I was able to complete the prediction with 98% accuracy and i saved it as pickle model. Now while trying to predict using this model I am ...
3
votes
0answers
48 views
How to do feature selection for clustering and implement it in python?
I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
0
votes
1answer
18 views
variables selection in regression models
I develop price prediction data model using multiple linear regression, ridge, lasso and elastic net regression, initially I had 215 variables. after creating models I ran a python code to check how ...
1
vote
1answer
31 views
How to extract features from long chemical names?
I have an interesting problem that I am uncertain about how to even get started. I am working on a binary classifier that will take a chemical name, encoded as a string, and predict whether it is a '...
0
votes
0answers
12 views
CNN: What's the relationship between point clouds and features derived from point clouds?
What's the relationship between point clouds and features derived from point clouds? Particularly in CNN prediction.
Particularly, I have point clouds about which I can imagine features that are ...
5
votes
2answers
367 views
RandomForest and tree feature importance in scikit-learn
What is the difference between model.feature_importances_ and tree.feature_importances_ in the following code:
...
0
votes
0answers
18 views
Encode features for Machine Learning Model
I am working on a classification problem on medical reports. I am taking ngrams as features. The problem is that there are few attributes that a single ngram can posses. For example, if 'abdominal ...
8
votes
1answer
276 views
Why continuous features are more important than categorical features in decision tree models?
I have both categorical and continuous features in my prediction model and want to select (and rank) most important features.
I have converted all categorical variables into dummy variables using one ...
0
votes
0answers
32 views
How to find feature names of Breast Cancer Wisconsin (Prognostic) Data Set?
I want to work on a data set from UCI machine learning repository, It's WPBC.
After reading the document on it website and some researching on the internet just understand that it has 34 features as ...
5
votes
1answer
235 views
Why ML model produces different results despite random_state defined? And how to set global random seed for sklearn
I have been running few ML models on same set of data for a binary classification problem with class proportion of 33:67.
I had the same algorithms and same set of hyperparamters during yesterday and ...
2
votes
2answers
41 views
Should I create a separate column for each Id value in a feature column or can I use the feature column as it is?
I am working on developing a model for predicting, revenue that a movie will make. One of the features in the training set contains id of the series that a movie belongs to.Say, Star Wars series has ...
0
votes
1answer
31 views
comparison of t-SNE and PCA and truncate SVD
How to compare the trucate SVD ,PCA, and T-SNE?
What we can say about features if t-SNE and PCA and truncate SVD digaram is in this figure?
2
votes
1answer
40 views
How to get the best combinations of features for a sale optimization problem?
I have a database of shoes items from the same brand with many variables (features) like the size, the color or the shape. I also have the produced and sold quantity for the last years.
This is a ...
0
votes
0answers
7 views
Dealing with built hierchical linear dependencies in my features
In a supervised learning problem, I am working on a dataset with a lot of hierachical linear dependencies between features. Let me be more clear about what I mean : In my dataset, there is a lot of ...
1
vote
3answers
79 views
Linear Regression finding best fit
I am trying to fit a LR model with an obvious objective to find a best fit. model which can achieve lowest RSS.
I have many independent variable so i have decided to yous Backward selection (We start ...
2
votes
2answers
58 views
Which feature selection technique to pickup(Boruta vs RFE vs step selection)
I have data with 103 columns. I would like to understand which algorithm is best for feature selection and what may be the logic to call any feature as best.
...
4
votes
4answers
109 views
How to automatically test for the best parameters for transformed independent variable in linear model
Let's assume that I have a linear model with $k$ variables:
$y = \beta_0 + \beta_1\cdot x_1 + \dots + \beta_k \cdot x_k$.
Now, I want to add variable $x_{k+1}$, but, according to domain knowledge, ...
2
votes
2answers
42 views
differences between feature weighting and feature selection
what are the differences between feature weighting and feature selection? And is feature importance like feature weighting?
1
vote
1answer
33 views
Threshold to consider new feature as a new finding to a model?
I am working on binary classification problem with 5K records and 60 features.
Through feature selection, I narrowed it down to 14 features.
In existing literature, I see that there are well-known 5 ...
2
votes
1answer
45 views
How to select best feature set and not ranking for tree based models?
I am currently using feature selection approaches like filter, wrapper, embedded etc.
All these methods give different set of ...
2
votes
1answer
389 views
feature selection using genetic algorithm in Python?
I have a dataset of 4712 records and 60+ features working on a binary classification problem. I already tried out all the feature selection approaches like ...
0
votes
1answer
53 views
1: 10 rule in logistic regression - EPV
I have a dataset with 4712 records. Label Yes - 1558 records and Label No - 3554 records.
I read online that ...
1
vote
3answers
89 views
How to handle features with very broad range
I have a long list of continuous values like in the image below:
The plot looks like this:
How to handle such features? If I train the model with this, the model will not have the best precision, ...
2
votes
1answer
56 views
Extremely high gain with LightGBM
I am working on a binary classification problem. The target variable is not linearly separable, so I've decided to use LightGBM with default parameters (I only play with n_estimators on range from 10 -...
2
votes
2answers
62 views
How to do feature selection after using pre-trained word embeddings?
I am working on a multiclass text classification problem. I want to use the top k features based on mutual information (mutual_info_classif) for training my model.
...
2
votes
1answer
126 views
How to interpret dummy variable in ML prediction?
I am working on a binary classification problem where I have a mix of continuous and categorical variables.
Categorical variables were created by me using ...
2
votes
1answer
27 views
How to justify a predictor in influencing the outcome?
I am working on a prediction (binary classification) problem
Currently I get an AUC score of 85-86 and F1-score of ...
1
vote
1answer
52 views
How to interpret shapley force plot for feature importance?
I am trying to practice and learn shapley value approach to explain my predictions on a binary classification problem. However am having difficulty in understanding the below plot.
1) Does it ...
2
votes
1answer
39 views
Prediction vs causation in a ML project
I am performing a classification task and was able to identify significant predictors (important features using Random Forest) that can help separate the classes or influence the outcome.
But I read ...
2
votes
1answer
86 views
How to interpret Shapley value plot for a model?
I was trying to use Shapley value approach for understanding the model predictions. I am trying this on a Xgboost model. My plot ...
2
votes
1answer
40 views
Logistic Regression Model for categorical features with multiple values in each category
I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
0
votes
1answer
132 views
Find the top n features from feature set using absolute values of `feature_log_prob_ ` parameter of `MultinomialNB`
I am working on Donors choose dataset and have converted categorical, numerical and text features into vectors. I want to find the top 20 features from my 5095 features using absolute values of ...
1
vote
1answer
31 views
how to determine percentage below which a feature can be removed from a model
Let feature $feat$ contain one value $A$ that occurs 5% of the time, while 95% of the time it is empty.
Instead of arbitrary saying features that have less than 5% should not be included into the ...
