Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
495
questions
2
votes
0answers
7 views
When does random forest feature importance fail?
I'm curious about the assumptions of random forest feature importance.
In this paper, the author says that
"We show that random forest variable importance measures are a sensible means for ...
1
vote
0answers
21 views
Linear Regression on data with bimodal outcome
I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
0
votes
0answers
23 views
how to use sklearn without feature selection
I am trying to study the effect of using feature selection onmy text classification code . I want to make a rating without any feature selection, but sklearn use document frequency (df) by default
...
1
vote
1answer
34 views
Association between features
Given the anonymized dataset of features below, where:
"code" is a categorical variable.
"x1" and "x2" are continuous variables.
"x3" and "x4" are extracted features. They are the mean values of ...
0
votes
0answers
42 views
How to use additional variables that are not available in test set?
I have additional variables in my dataset that are somewhat correlated to the continuous target variable, but that are completely unavailable in the test set. So, I'm wondering how the best to use ...
0
votes
1answer
47 views
Increase accuracy of classification problem [closed]
I am trying to build a classifier that predicts the compiler given some operations of assembly code. Here is the pandas dataframe:
What I do is using a TfidfVectorizer and select the features that ...
0
votes
0answers
13 views
Can I merge levels or factors having equal mean in categorical variable
I compared levels of categorical variable by their respected mean, obtain from continuous response variable using pivot table. I found that some of the levels is having nearly equal mean e.g 'BrDale' ...
1
vote
0answers
20 views
Extracting Features for Graph transformation
Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type.
The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
0
votes
1answer
53 views
Building an efficient feature vector
I am building a classifier for malware analysis, which predicts if I have a malware by looking at the intructions of an assembly code, such as push, mov,... and predicting the optimization method. ...
0
votes
0answers
16 views
Information Gain & Gini Index for NLP
I know how Information Gain and Gini Index work in General.
I have problem figuring out how to apply these techniques in NLP and text feature extraction.
Can someone show me an example of how to ...
0
votes
1answer
26 views
Problem building a feature vector
I am trying build a classifier for malware analysis for which basing in the instructions of an assembly code, such as push, mov,... I want to predict the compiler, and in a second time the ...
0
votes
1answer
32 views
Find the most relevant columns for each single class in pandas
The following question (this one) did not help me.
I have a big dataset, and I want to know which Columns are the most relevant for the Target Variable. I know that, in my case, for each class in the ...
0
votes
0answers
17 views
Find out which attribute of a movie causes the most variation in score
I'm tinkering around with a subset of the IMDb dataset, and have been thinking about what specific attributes of a movie impact its user rating. I am looking to survey what methods can be used to find ...
0
votes
0answers
7 views
Recursive feature elimination on train data or complete dataset and dummy encoding
I am using RFE with logistic regression. I will also be doing cross validation with RFE (RFECV in sklearn) to get the optimum number of features.
I am not sure whether to use RFECV on just train ...
0
votes
0answers
24 views
if I got feature importance of xgboost/LightGBM what is next?
If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information?
Is it better to just use the top n features and retrain the model?
Does the feature ...
1
vote
1answer
18 views
My models performs better with the arbitrary random feature. How can I interpret this?
I am training 6 different classifiers 'Decision Tree', 'Random Forest', 'Logistic regression' and 'SVM' with different kernels. There are about 80 dependent variables including categorical and ...
0
votes
1answer
19 views
How to choose the features for an algorithm from the given attached screenshot?
How to choose the features from the given attached heat map & correlation factor for the classification algorithm?
I have 6 different features i.e., ac233fc01403, ac233fc02eaa, ac233fc015f6, ...
1
vote
0answers
21 views
Increase accuracy of occupancy prediction?
I have a project that's aimed to predict the amount of occupants at my local gym given the date and weather.
Here's my Kaggle kernel
I have two datasets, occupants on a given hour and weather on a ...
1
vote
1answer
29 views
Implicit feature selection
I have heard that Random Forest and other tree based machines apply some kind of implicit feature selection.
My Question is: Does this also apply for machines like the SVM? As far as I understand is ...
0
votes
2answers
28 views
How to deal with a feature that has lot of categorical values?
I know this question has been asked before and I have tried a few things but those things are not working as expected for my usecase.
I have a 500 length feature vector. One of these features is a ...
2
votes
1answer
19 views
(Feature selection) In which cases it is legitimate to remove features manually?
I am dealing with the feature in which only one category takes up about 90%, the instances of more than 30 other categories are sparse.
Is it reasonable to remove this feature before building an ...
1
vote
1answer
36 views
(Feature Selection) different results from L2-based and Tree-based
I am doing feature selection using Sklearn:
Tree-based feature selection : RandomForestClassifier.feature_importances_
L2-based feature selection: LogisticRegression.coef_
Target variable is binary ...
2
votes
2answers
83 views
What do you call a feature that always has the same value?
Is there a standard term for a feature that always has the same value, i.e. that can be discarded without loss of information?
For example I am trying to classify cats vs dogs, and every example in ...
2
votes
1answer
60 views
Feature selection is not that useful?
I've been doing a few DataScience competitions now, and i'm noticing something quite odd and frustrating to me. Why is it frustrating? Because , in theory, when you read about datascience it's all ...
1
vote
1answer
85 views
How to install boruta in conda?
I want to install boruta in my anaconda environment, but if I execute
conda install boruta
It displays,
...
1
vote
2answers
53 views
Improving classifcation when some are less represented?
I have a multi-class classification problem. It performs quite well but on the least represented classes it doesn't. Indeed, here is the distribution :
And here are the classification results (I took ...
3
votes
0answers
27 views
Features selection with a lot of dummy variables in R
I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
0
votes
1answer
17 views
Pearson correlation make disapear the target column
I have a dataframe with some numerical and categorical values. I want to do some feature selection to visualise a low-dimensional split in the dataset when the target variable is the grade. Yet, when ...
2
votes
1answer
93 views
Feature Importance
I have a dataset with 10 features.
I've computed the feature importance using permutation importance with cross-validation from eli5, after fitting an extremely randomized trees (ET) classifier form ...
5
votes
0answers
59 views
Features reduction for the not correlated data set
I am working with classification problem on a training data set, which have 100 features. All the features in pairs haven't visible correlation. One can see it in the example pair plot for the some of ...
2
votes
1answer
29 views
Feature extraction in audio spectrogram
I have audio and its spectrogram of the words in English language. (A spectrogram is a frequency domain representation of a signal) Consider the words: chain, change, chair, chapter. As you can notice,...
0
votes
1answer
34 views
Feature selection, is it possible to combine wrapper and embedded methods?
I'm using neural network to predict PM10 concentration (a regression problem). Since the wrapper method is dependent on the model, so passing the neural network model that's optimized for all the ...
0
votes
1answer
59 views
xgboost feature selection and feature importance
when you create the new feature for data analysis
for linear regression, it is clear that the feature has to be linear with other features is better
but for xgboost what is the guideline to make a ...
1
vote
0answers
17 views
Difference between RFECV and SFS?
I used scikit.REFCV and mlxtend.SFS (backward) on the same data, same classifier, same cv, same scorer,... I also did a third version with sample weights passed to SFS's estimator
And i'm conflicted ...
0
votes
1answer
42 views
What variance threshold to consider in feature selection?
Consider a numerical dataset with continuous variables, that has been scaled to end up with values in the [0,1] range. How can I compute a reasonable variance threshold for all the variables?
1
vote
0answers
21 views
Scikit-compatible network lasso implementation
Is anyone aware of a scikit-compatible network Lasso (nLasso) implementation?
These papers have source code as well:
D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and ...
0
votes
0answers
76 views
Feature selection for string columns
I tried sklearn SelectKBest for feature selection (feature scores) of supervised machine learning before. It looks good, but can only take numbers as inputs.
Although I can present some string ...
1
vote
0answers
135 views
Explanation of how DeepExplainer works to obtain SHAP values in simple terms
I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
1
vote
2answers
96 views
Finding Feature Importance in CNN's?
Let's say I have images of cars. For each image in the dataset, I have let's say 3 pictures of the same car but in different angles.
1) The first image is the picture of the car from the front.
2) ...
0
votes
0answers
24 views
What is the dimension reduction method to large numbers of independent features while only two of them are important?why?
What is the dimension reduction method to model a data with large numbers of independent features (for instance 5k features), while only two of them are important (are effective in cost function)?
I ...
0
votes
0answers
19 views
Multi-level data, what is the best approach?
I'm working on a dataset and having some problems. I hope you can give me your insight.
So my objective is to predict customer churn based on incidents. Each incident is related to a contract which ...
3
votes
2answers
41 views
Decision Trees Should We Discard Low Importance Features?
I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_.
In my model out ...
2
votes
3answers
85 views
Can ridge regression be used for feature selection?
I'm trying to figure out whether using Ridge Regression for regularization can be used to cause a more sparse hypothesis however to me it seems like ridge will never actually bring any coefficients to ...
1
vote
2answers
33 views
Feature selection filter methods
I am confused about when to use which filter methods for feature selection. I tried to learn them through online resources and found methods like chi-square, variance threshold, F-test, Mutual ...
2
votes
1answer
107 views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
2
votes
2answers
94 views
Using random forest for selecting variables returns the entire dataframe
I am in the process of dimensionality reduction. I am using Random Forest to find the columns with the highest
level of correlation with the target SalePrice column.
The problem is that the output ...
0
votes
1answer
43 views
How to input a 3d model into ML algorithm?
I have a machine learning model that uses csv with measured data about buildings: width, length, height etc. I use it to predict some features and it works properly.
I would like to drop csv with ...
1
vote
1answer
155 views
Light GBM Regressor, L1 & L2 Regularization and Feature Importances
I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances.
Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
0
votes
0answers
24 views
Feature selection for circular data in time-series
I'm predicting ozone concentration based on meteorological and seasonal variables. In the feature engineering stage I converted the MONTH, DAY_OF_WEEK, DAY_OF_YEAR to its sin and cosine components ...
2
votes
1answer
137 views
How does SHAP values help us to determine importance of a feature for a model trained by gradient boost?
I've read http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf and https://medium.com/@gabrieltseng/interpreting-complex-models-with-shap-values-1c187db6ec83 which ...
