Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
829
questions
0
votes
1answer
35 views
Find most important and least important features for clustering algorithm
I am experimenting with clustering algorithms, like K-Means.
Right now, I use all variables as input for the clustering algorithm.
I am wondering if it is appropriate to do feature selection for ...
2
votes
0answers
23 views
Random Forest importances vs Feature Permutation: cummulative sum of importances are 1 and 0.1, respectively. Make sense?
I am performing feature selection by using two methods: MDI (RandomForest importances) and Feature Permutation, in order to compare what are the features considered relevant for both methods. My ...
1
vote
0answers
24 views
How to implement kfold and cv into Hybrid feature selection and evaluate the classification model performance?
I have been working on a Hybrid feature selection combined with hyperopt package for hyperparameter tuning and I am thinking about evaluating the performance of several model classifiers. I looked ...
1
vote
1answer
16 views
How should I engineer features for Named Entity Identification task?
I was working on Named Entity Identification (not recognition) task. In this NLP task, given a sentence, model has to predict whether each word (aka token) is named entity or not. The dataset used ...
1
vote
0answers
36 views
Distance calculation for nonlinear features
Dear Data science community,
Please see the attached. I plotted my data using t-SNE. In the figure, group A and B are 100% separable with random forest model. I want to calculate the distance of ...
0
votes
1answer
25 views
Why scikit-learn's sequential feature selection requires how much features to be selected beforehand?
From the version 0.24, the scikit-learn has new method 'SequentialFeatureSelector', which adds (forward selection) or removes (backward selection) features to form a feature subset in a greedy fashion....
1
vote
1answer
24 views
How many features should be there in a dataset to apply any feature selection method?
I am working on a time series, regression problem, where I have 10 features and 180 observations. I would like to understand what the minimum number of features should be in a dataset to use feature ...
0
votes
1answer
29 views
Steps of multiclass classification problem
So this question is more theoretical, than a practical one. I got a dataframe with 4 classes of cars' body types (e.g. sedan, hatchback, etc.) and different characteristics (doors, seats, maximum ...
0
votes
0answers
10 views
How to perform feature selection on a dataset using correlation-based feature selection process
I have a dataset and on that, I have to perform feature selection using a correlation-based feature selection process (using scikit-learn), can anyone please show me how to do it with a small example ...
1
vote
2answers
23 views
When Does Feature Selection Takes Place?
I have a dataset where there are categorical features as well as numeric features, and I have to perform OneHotEncoding, Normalization and feature selection on it.
In what order should I perform these ...
0
votes
1answer
32 views
Best Way to find the important features for the model [duplicate]
I have data with 245 Features and almost all of the features are categorical. I would like to know what will be the best approach to find the important features for training the model. I know I can ...
0
votes
0answers
6 views
Best Practices For Dealing With This Scenario
I'm presently building a spam classifier. The model is unable to even overfit the training set at present. To investigate, I plotted the distributions of the model's features, and compared them across ...
0
votes
0answers
7 views
Xgboost train using gamma(feature selection)
I'm training xgb model to predict binary target (classification problem).
I set gamma parameter little bit high to alleviate overfit. As a result only 75% of initial feautures have been selected in ...
0
votes
0answers
9 views
Creating a popularity index from multivariate data
I have some data from an ecommerce website with features like product_name, product_category product_link, product_id, free_delivery(1 or 0), price, discount, avg_rating, number of reviews, ...
0
votes
0answers
4 views
Feature selection algorithm for psychometrics, when there is several predicted variables
I'm on a psychometric study. It is a survey.
All variables are on a scale of 7. So these are considered as continuous variables.
I have this dataset:
600 features
100 predicted variables
100 survey ...
0
votes
0answers
16 views
How to handle both the categorical and ordinal features in a single data sets?
I was practicing Lasso regression with the SPARCS hospital dataset. There are two kinds of features in the dataset:
Categorical features like location of the hospital, demographics of patients, etc.
...
0
votes
0answers
28 views
Interpret messy SHAP summary plot for a Random Forest
I have the following plot and it shows SHAP values in a way I haven't seen anywhere. I have a small dataset (400 observations). I used SHAP for my other models and those have the more traditional ...
3
votes
1answer
42 views
When should I oversample data?
I am dealing with multi-class classifiers. My data is unbalanced. Hence, I need to apply sampling techniques before training (undersampling or oversampling). When I apply undersampling, ...
3
votes
2answers
51 views
Is it possible to 'group features' for a decision tree model?
At each node of a decision tree, we must choose a collection of features to split along.
Suppose we know a priori that the features can be partitioned into subsets that are 'correlated', i.e. this ...
1
vote
0answers
24 views
Permutation Importance is 0 with high accuracy
I am using sklearn's permutation_importance for feature selection, and all my features return score decreases of 0, even though my model accuracy is 0.96.
I have ...
0
votes
1answer
14 views
Method of choosing features for better clustering?
I'm working on a project where I need to cluster data. After doing all the usual steps (in no distinct order: one-hot/BaseN encoding categorical data, doing a Quantile Transform due to none of the ...
0
votes
1answer
28 views
BorutaShap implementation
I want to use BorutaShap for feature selection in my model. I have my train_x as an numpy.ndarray and I want to pass it to the ...
0
votes
0answers
22 views
Searching for implementations of multilabel feature selection
Does anyone know of any packages that implement multilabel feature selection algorithms? There are many papers introducing algorithms, but few if any seem to be implemented publicly. I have been able ...
2
votes
1answer
49 views
Logistic Regression to model a rare event
I have a data set which has data on consumers and a flag for whether they have expressed interest in a product or not. I am looking to build a model using R which will be able to predict whether or ...
0
votes
0answers
20 views
Features importance in model
I've been using azure's auto ML platform for a couple of weeks now and recently I've trained a model and came across a strange looking aggregate feature importance chart in the explanations tab. The ...
2
votes
1answer
102 views
Ongoing feature selection
If you have a set of n features you have 2^n-1 non-empty feature subsets. As a result, if you pick one of them you are unlikely to have found the best one.
To me, it seems intuitive that as you build ...
0
votes
0answers
12 views
Feature selection with mixed categorical and numerical data
It is well known that when you do feature selection, there are statistical tools for comparing features of the same type if you know the output, for example
Numerical data - > numerical output: ...
3
votes
1answer
106 views
What is the point of generating new features (linear or non linear) out of existing features in a dataset?
During feature engineering, we can create new features out of existing ones by using arithmetic operations albeit linear or not.
Let's say we have two features x and z. We can then create (engineer) a ...
1
vote
1answer
43 views
Can the feature importance scores of multiple ML algorithms be combined?
Multiple Machine Learning algorithms are developed to rank some features. Is there an algorithm or statistical approach that can combine the ranking of each of these features into a final ranked list ...
0
votes
0answers
16 views
EDA on multivariate time series data
I'm working with a medical dataset in a "long" format, meaning each row represents a timestamp and I have a column named "test" representing the test, an additional column ...
0
votes
0answers
20 views
Should Chi Squared test for feature selection be applied on train dataset or the whole dataset?
I am working on building a logistic regression model. I am planning to run chi squared test for feature selection. Should I run it on train dataset or the whole dataset?
1
vote
1answer
28 views
In Text Classification if I get similar performance with 100 features and 200 features, which model should I go ahead with?
I have built two text classifier models, one has 200 features the other has 100 features (reduced to 100 from 200 after feature selection). I see similar performances in both. Which model should I go ...
0
votes
1answer
23 views
Find correlation within vectorised data
I am at the feature selection phase of my project but I have my vectorised data. Is there a way to find highly correlated features and then remove them? After this I would then like to remove features ...
0
votes
1answer
23 views
Creating a complex featureset for regression modeling
I am currently working a on project that requires me to convert all of the categorical variables to continuous (or binary) variables to build a regression model. The problem is that I have more than ...
1
vote
0answers
13 views
Rule of Thumb for number of observations required to train a model with n independent variables?
I am aware adding more features to a model leads to overfitting of a model. Is there a rule of thumb for minimum number of rows required to build a model with n features in order to build a ...
0
votes
0answers
6 views
Any reliable dimension reduction implementations available to address class overlapping scenario?
I am currently resolving a class overlapping problem in machine learning and while running some class separation experiments I have observed that Linear Discriminant Analysis (LDA) is able to perform ...
0
votes
0answers
6 views
Handling High dimensionality datasets for EDA
I have a dataset consists of 500k rows and around 500 variables initially. And I would like to run EDA among these, then modeling of course.
But this high dimension should be reduced of course. First ...
0
votes
2answers
52 views
difference between feature effect and feature importance
Is there a difference between feature effect (eg SHAP effect) and feature importance in machine learning terminologies?
1
vote
0answers
28 views
feature selection for categorical variables
I have been working on this issue for quite a while and going nowhere. If I have categorical features in my dataset and some of them have high dimensions, if I OHE them, I get a dataset with high ...
0
votes
0answers
18 views
Feature importance not providing score for each feature
I have a data frame of this shape (808616, 10744) and I use this model
...
2
votes
1answer
37 views
Feature selection for two seperate datasets
Currently, I'm doing research with experimental data. The data comes from two experiments with two slightly different tasks, but with the same setup in a VR environment. Both experiments were done ...
1
vote
0answers
17 views
Creating sub categories
I have data we have collected quarterly over the last two years from two organisations. They are collected via the use of 29 questions. For each organisation, there are about 500 answers per question. ...
0
votes
0answers
17 views
Feature Engineering on 3 dimensions data
I'm doing a task where I was given 3 features (a1, a2 and a3) and 3 heavily unbalanced classes. I tried many balancing techniques like SMOTE and undersampling. None of them gives me a reasonable ...
0
votes
0answers
8 views
Feature set choice in Google's Vertex AI/AutoML
This is a subjective question on utilizing Vertex AI/AutoML in practice. I posted it on stackoverflow and it was closed. I hope it is within scope here.
I'm using Google's Vertex AI/AutoML's Tabular ...
1
vote
1answer
30 views
How to interpret .get_booster().get_score(importance_type='weight') for XGBRegressor()
I am trying to do feature selection using XGRegressor(). I am doing this because I have many features to choose from over 4,000. Once I have a set of features I have a neural network I created to use ...
1
vote
2answers
37 views
what would be the correct representation of categorical variables like sex?
I have a doubt about what will be the right way to use or represent categorical variables with only two values like "sex". I have checked it up from different sources, but I was not able to ...
0
votes
1answer
22 views
Would it be a good idea to use PCA output as input in models?
I have some dummy variables that indicate the occurrence of an event. There is so many of them, so I used PCA on them, and it appears some of them are rather correlated together.
Would it be a good ...
0
votes
0answers
8 views
Generating discriminative features and addressing feature variations across different datasets for same set of data variables?
I am trying to build a classification-based machine learning model using efficient feature selection methods that predict the label of a dependent variable based on few independent variables.
The ...
0
votes
1answer
116 views
Is feature importance in XGBoost or in any other tree based method reliable?
This question is quite long, if you know how feature importance to tree based methods works i suggest you to skip to text below the image.
Feature importance (FI) in tree based methods is given by ...
1
vote
1answer
17 views
How to find the probability of a word to belong to a dataset of text
I have two text datasets, one with people that have a certain medical condition and another of random patients. I want to figure out what words are more likely to show up in the dataset with that ...
