Data Science

Questions tagged [feature-selection]

Ask Question

Methods and principles of selecting a subset of attributes for use in further modelling

587 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
-1
votes
0answers
9 views

How to select best features SVM- numerical inputs and categorical output

I have a number of features and I want to reduce the dimensionality to ensure good model accuracy. How do I select the best features where all the inputs are numerical and the outputs are categorical. ...
0
votes
0answers
9 views

Variable with extra small Pearson coefficient has bigger positive impact on ML model performance than variable with bigger Pearson

I made some machine learning models using Python sci-kit learn library and I found some strange situation for me regarding the real importance of some variables (features) to the ML model. I found ...
1
vote
0answers
14 views

Feature engineering one step at a time or in bunches?

Currently, I'm working on my very first classification project. If you want to know what dataset I'm working with, think "playing stairway to heaven in your local guitar store", and it will probably ...
0
votes
1answer
24 views

Multi-class classification with only one feature

I am studying the efficacy of using a single feature for predicting a set of events (which is a multi-class classification problem). I was wondering if it makes any sense to use only one feature for ...
1
vote
1answer
29 views

using feature selection to improve model performance

I have a highly sparse dataset that I am using to predict a continuous variable via a random forest regression. I have achieved an acceptable level of performance following cross-validation, and I am ...
1
vote
2answers
32 views

Different approaches to label data

I have a dataset of patient records but they don't have labels. I would like to label them and would like to know what are the different approaches available that I can consider to label them. For ...
0
votes
3answers
31 views

Why to exclude features used for label generation during modeling?

I have a dataset like below without labels But with the help of experts opinion, we generate labels based on the below 3 rules (all 3 rules has to be met to label it as 1) So now the dataset looks ...
0
votes
1answer
25 views

How to create a feature vector given final set of feature maps?

I've got a faster-rcnn (resnet-101 backbone) for object detection, and am extracting feature tensors for each detected object, which is a 7x7x2048 tensor (basically 2048 feature maps, each 7x7). For ...
0
votes
1answer
21 views

What are some significance tests to rank features(multiple) before training the data

I have 8 features for a classification problem. The target value tells if there was an anomaly or not. I want to run some significance tests to rank each feature, as being a distinctive feature of ...
1
vote
2answers
43 views

Scikit-learn OneHotEncoder effect on feature selection

If I need to run feature selection on my dataset isn't it problematic to use OneHotEncoder? Couldn't it then decide to remove a one of the encoding columns? How should I deal with this? Thank you.
2
votes
0answers
32 views

Feature selection for data with both continuous and categorical features?

I am working on a classification problem with 4 ordinal classes to predict, labelling/predicting samples as either a number from 1-4. My training dataset has 284 features by ~40,000 samples and I am ...
1
vote
1answer
17 views

In feature selection, I came across a situation where NaN were filled by median of the column values

Why the median value is used for NaN? Why not something else like mean? What is the logic behind using the median value?
1
vote
0answers
26 views

Why not use constant instead of permutation for model agnostic predictor importance?

I want to determine predictor importance. Ideal is to re-train same model on same dataset missing each variable in turn. This is too time consuming. The recommendation I have seen everywhere is to "...
1
vote
0answers
19 views

Does EDA helps only in case of linear regression?

I know what Explanatory data analysis is and how it helps us investigate and understand the data. What I dont understand is how does this help in case of nonlinear relationships? I mean if I'm using ...
2
votes
0answers
21 views

Which stage should the correlation analysis be done?

I was thinking about it, but I couldn't find a logical explanation. Mostly im following below steps after data become ready: Correlation analysis and elimination Apply dummy if categorical variables ...
2
votes
1answer
39 views

Random Forest workflow?

I have a data-set comprised of a fairly large number of columns (over 1000) relative to the number of rows (370) that I am currently running a random forest regression on. I am a little confused with ...
1
vote
1answer
18 views

How to do backward features elimination when considering interactions between them

I have a multi linear regression problem, $Y$ is my target and $X_1, X_2, X_3$ are my features. In my regression, I consider the interaction between $X_1, X_2, X_3$ and I add a bias. So my problem ...
1
vote
0answers
12 views

What toolbox to use to create multi-output random forest(reggression) with custom spltting function at each node?

I am trying to implement "Real Time Head Pose Estimation fromConsumer Depth Cameras" by Fanelli et al. I need to train a random forest(regression) with the following criterion The predicted output is ...
1
vote
1answer
41 views

Correlation based Feature Selection vs Feature Engineering

I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa. Let's say I just want to get the best possible performance on a couple of models like a neural ...
1
vote
1answer
72 views

How to get feature importance from a keras deep learning model?

In case of scikit-learn's models, we can get feature importance using the relevant attributes of the model. I've been working on a RNN, using LSTMs for text embedding. Is there any way to get ...
1
vote
0answers
20 views

Model-independent measures for feature importance given highly correlated features

I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
0
votes
1answer
29 views

What is the name of this statistical interaction?

What is the name of the following statistical / informational interaction: given A, I know exactly what B is. given B, I know to some extent what A is. I'm not looking for a probability but rather ...
1
vote
0answers
13 views

Do you need to perform variables reduction for tree-based models?

I know for methods and linear regression, GLM, Logistic regression, we typically run through a lot of variable reduction methods, i.e, forward/backward/stepwise, univariate analysis; variable ...
5
votes
3answers
488 views

What can be done with highly correlated variables (>.95 and <-.95)

I hope we can remove the highly correlated variables based on the feature importance may be with PCA etc. Is there anything we can do with highly correlated variables/ Thanks in advance !
-3
votes
1answer
29 views

How to do feature selection for classification problem? Which technique will work? [closed]

I have 200 variables with 200000 records. How to find best features from this variables? I have tried correlation technique via Heatmap but all the variables have near to same correlation score < 0....
1
vote
0answers
16 views

How do decision tree works for feature selection?

I have a dataframe with a feature selection problem. I want to get the variables explaining the variance within each segment of the following dataset: ...
0
votes
0answers
9 views

Sequential Forward Selection (SFS) for standard Feed Forward Neural Network

I'm comparing the classification performance (accuracy, f1-score etc.) of several predictive models (logistic regression, random forest, xgboost etc.) with a standard feedforward neural network. For ...
1
vote
1answer
38 views

How to use the $\chi^2$ test to select the features, that can be String or categorical?

I want statistics to select the characteristics that have the greatest relationship to the output variable. Thanks to this article, I learned that the scikit-learn ...
1
vote
0answers
22 views

How can we assess the importance of the features even if we ended up applying PCA?

There are multiple techniques to analyze the feature importance (permutations, SHAP values, etc). It is essential that, in order to improve the interpretability of the model, we can somehow map the ...
2
votes
0answers
13 views

stacking features vs concatenating layers

I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
2
votes
0answers
12 views

features to help distinguish between document images

we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
2
votes
1answer
24 views

Several independent variables based on the same underlying data

I got a data containing, among others, two feature variables, which are based from the same underlying data (i.e. have mutual information), but they convey different information/message. How to handle ...
1
vote
0answers
14 views

How to do Multivariate Adaptive Regression Splines feature selection in python? Specifically, I need the python equivalent of the earth function in R

This is the code in R: marsModel <- earth(eval(parse(text=paste(ResponseVariable,"~."))), data = data) #build model ev <- evimp (marsModel) Response ...
0
votes
1answer
115 views

Feature Importance from a GridSearchCV

I created a GridSearchCV for a Random Forest Regressor. Now i want to check the feature importance. I searched around and I found this ...
3
votes
1answer
45 views

Should Feature Selection processes be apply on training data or on all data?

I've realized that on examples and guides, sometimes feature selection processes (correlation elimination, backward/stepwise) are applied on the train data after splitting all data but on the other ...
1
vote
0answers
46 views

How to predict using the pickle model?

I am new to data science and trying to learn something. I was able to complete the prediction with 98% accuracy and i saved it as pickle model. Now while trying to predict using this model I am ...
3
votes
0answers
140 views

How to do feature selection for clustering and implement it in python?

I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
0
votes
1answer
23 views

variables selection in regression models

I develop price prediction data model using multiple linear regression, ridge, lasso and elastic net regression, initially I had 215 variables. after creating models I ran a python code to check how ...
1
vote
1answer
33 views

How to extract features from long chemical names?

I have an interesting problem that I am uncertain about how to even get started. I am working on a binary classifier that will take a chemical name, encoded as a string, and predict whether it is a '...
0
votes
0answers
15 views

CNN: What's the relationship between point clouds and features derived from point clouds?

What's the relationship between point clouds and features derived from point clouds? Particularly in CNN prediction. Particularly, I have point clouds about which I can imagine features that are ...
5
votes
2answers
467 views

RandomForest and tree feature importance in scikit-learn

What is the difference between model.feature_importances_ and tree.feature_importances_ in the following code: ...
0
votes
0answers
18 views

Encode features for Machine Learning Model

I am working on a classification problem on medical reports. I am taking ngrams as features. The problem is that there are few attributes that a single ngram can posses. For example, if 'abdominal ...
8
votes
1answer
321 views

Why continuous features are more important than categorical features in decision tree models?

I have both categorical and continuous features in my prediction model and want to select (and rank) most important features. I have converted all categorical variables into dummy variables using one ...
0
votes
0answers
36 views

How to find feature names of Breast Cancer Wisconsin (Prognostic) Data Set?

I want to work on a data set from UCI machine learning repository, It's WPBC. After reading the document on it website and some researching on the internet just understand that it has 34 features as ...
5
votes
1answer
287 views

Why ML model produces different results despite random_state defined? And how to set global random seed for sklearn

I have been running few ML models on same set of data for a binary classification problem with class proportion of 33:67. I had the same algorithms and same set of hyperparamters during yesterday and ...
2
votes
2answers
44 views

Should I create a separate column for each Id value in a feature column or can I use the feature column as it is?

I am working on developing a model for predicting, revenue that a movie will make. One of the features in the training set contains id of the series that a movie belongs to.Say, Star Wars series has ...
0
votes
1answer
45 views

comparison of t-SNE and PCA and truncate SVD

How to compare the trucate SVD ,PCA, and T-SNE? What we can say about features if t-SNE and PCA and truncate SVD digaram is in this figure?
2
votes
1answer
50 views

How to get the best combinations of features for a sale optimization problem?

I have a database of shoes items from the same brand with many variables (features) like the size, the color or the shape. I also have the produced and sold quantity for the last years. This is a ...
0
votes
0answers
7 views

Dealing with built hierchical linear dependencies in my features

In a supervised learning problem, I am working on a dataset with a lot of hierachical linear dependencies between features. Let me be more clear about what I mean : In my dataset, there is a lot of ...
1
vote
3answers
93 views

Linear Regression finding best fit

I am trying to fit a LR model with an obvious objective to find a best fit. model which can achieve lowest RSS. I have many independent variable so i have decided to yous Backward selection (We start ...

1
2 3 4 5
…
12