Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
964
questions
6
votes
2
answers
2k
views
Metrics to evaluate features' importance in classification problem (with random forest)
I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest.
The most widely used ones apparrently are:
Cross Entropy-Information ...
CommunityBot
- 1
0
votes
0
answers
26
views
Multiple Hypothesis Testing in feature selection process
I am doing feature selection of features which are of binary nature i.e. each feature represents presence or absence of a substructure in a molecule. And I have a target variable of two classes. My ...
- 47
0
votes
1
answer
334
views
BorutaShap implementation
I want to use BorutaShap for feature selection in my model. I have my train_x as an numpy.ndarray and I want to pass it to the ...
CommunityBot
- 1
0
votes
1
answer
822
views
How do feature selection on a sparse matrix?
Say I want to do features selection on a sparse matrix, i.e., 10,000 rows x 1500 features, but the matrix is mostly sparse. Let's say the features are all numeric and the target is binary and discrete....
CommunityBot
- 1
3
votes
1
answer
91
views
Determine the most important documents for supervised learning
I have somewhat of a general/high level question.
Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
CommunityBot
- 1
0
votes
1
answer
499
views
Feature engineering: The more features I add the better RMSE I get?
I have a model with 7 features, I'm trying to figure out if I can improve the performance of this model by adding additional features. So I'm relying on the RMSE to measure the accuracy of my ...
CommunityBot
- 1
0
votes
1
answer
18
views
Is there a standard data science workflow/decision tree?
I'm looking for some kind of reference that essentially shows an example of an entire data analysis workflow beginning with feature engineering, and ending with analyzing the results.
I know the ...
- 23.8k
1
vote
1
answer
548
views
Feature engineering before splitting
This is a sister post to the original closed post (here). Since the data transformation part is done after data spliting on the TRAINING data only, I wonder wouldn't such transformation has dependency ...
CommunityBot
- 1
0
votes
1
answer
172
views
Change feature importance in a trained model
I am giving a toy example for describing a real world business problem. Let's say I am a publisher and I have some book stores to visit. By visiting those stores I will check whether they have ...
CommunityBot
- 1
0
votes
0
answers
28
views
Feature selection: ANOVA between features vs within a feature
I am currently performing feature selection on a dataset containing continuous and categorical features. The target is a continuous variable.
If I understand properly, ANOVA can be used between ...
- 1
0
votes
0
answers
12
views
Feature selection on datasets with both categorical and numerical features
I'm proposing a novel methodology for feature selection in the context of tabular datasets that contain both numerical and categorical features. In order to prove the efficacy of my methodology, I ...
0
votes
1
answer
311
views
Information Gain & Gini Index for NLP
I know how Information Gain and Gini Index work in General.
I have problem figuring out how to apply these techniques in NLP and text feature extraction.
Can someone show me an example of how to ...
CommunityBot
- 1
0
votes
1
answer
74
views
Creating a complex featureset for regression modeling
I am currently working a on project that requires me to convert all of the categorical variables to continuous (or binary) variables to build a regression model. The problem is that I have more than ...
CommunityBot
- 1
0
votes
2
answers
657
views
Regression performance with Feature Selection
I would like to ask you a theoretical question. In my project I am trying to get a better performance from my regression model by feature selection methods, especially with CatBoost feature ...
CommunityBot
- 1
1
vote
1
answer
98
views
Literature on selecting specific dimensions in a word embedding vector
I am aware that the different dimensions in the word embedding represents different information and algebraic operations can be performed between two embeddings for example.
Can anyone point me to ...
CommunityBot
- 1
0
votes
1
answer
37
views
Why the order of the fearures affects synapse LightGBM predictions?
I am using LighGBM Classifier and Regressor and it seems that the order of the features I am adding, affect the predictions of the model. Everytime I change the order, another result comes up and with ...
- 121
1
vote
0
answers
16
views
4
votes
1
answer
1k
views
SelectKBest and Correlation returns me excatly same feature selection. How?
Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
CommunityBot
- 1
2
votes
1
answer
259
views
Logistic Regression Model for categorical features with multiple values in each category
I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
CommunityBot
- 1
0
votes
1
answer
76
views
Correlation with target variable for regression problem
Given the following dataframe
age job salary
0 1 Doctor 100
1 2 Engineer 200
2 3 Lawyer 300
...
with ...
CommunityBot
- 1
0
votes
2
answers
465
views
How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?
Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
- 156
1
vote
1
answer
393
views
Does sklearn perform feature selection within cross validation?
I would like to add a feature selector on my pipeline and use gridsearchcv to tune both the hyperparameters of the selector and the classifier(s).
I am wondering if sklearn performs feature selection ...
CommunityBot
- 1
0
votes
0
answers
16
views
Feature selection for siamese network
I have a regression problem for which two observations are compared by a siamese-like Multilayer Perceptron.
Each observation 'O' is described by a feature vector 'X' of a certain number 'N' of ...
1
vote
1
answer
176
views
Feature importance has more variables than included in .csv?
I have a .csv dataset with 26 variables, ranging from Age to Weight and so forth. I plotted a feature importance plot with;
...
CommunityBot
- 1
0
votes
0
answers
7
views
Feature selection methods when input data is continuous but target variable is categorical
I plan on extracting features from a univariate time series, and use a feature selection method to select relevant features to predict a binary target variable via logistic regression. But, I have 2 ...
0
votes
1
answer
75
views
Should I get dummies and then look at multicollinearity?
I have data that includes continuous and categorical features. The task is regression and I am looking to remove features that are high correlated with other features (multicollinearity). To do this, ...
CommunityBot
- 1
1
vote
1
answer
106
views
Minimum number of features for Naïve Bayes model
I keep on reading that Naive Bayes needs fewer features than many other ML algorithms. But what's the minimum number of features you actually need to get good results (90% accuracy) with a Naive Bayes ...
CommunityBot
- 1
0
votes
1
answer
165
views
How to use hierarchical variable in a ML model
I am working on a binary classification problem with 1000 rows and 20 variables.
I have variables like product_id, city, ...
CommunityBot
- 1
2
votes
1
answer
898
views
Is there a way to output feature importance based on the outputted class?
I'm running a random forest classifier in Python (two classes). I am using the feature_importances_ method of the ...
CommunityBot
- 1
0
votes
1
answer
196
views
Percentage of missing values so that we can't perform imputation
(This is not a question on ways to handle missing data)
I have a dataset with around $80$ or so features and around $100000$ rows. Several of those features have missing (NULL) values for a "large ...
- 364
0
votes
0
answers
12
views
How is it called when instead of creating predective models finding patterns in observed data (ML) you tried to guess the model theorically...?
I'm a college student appasionated of machine learning and I've decided to my bachelor thesis about it. I thought that as an interesting introduction to machine learning, I could introduce it by ...
2
votes
1
answer
343
views
How to interpret a specific feature importance?
Apologies for a very case specific question. I have a dataset of genes, with which I am using machine learning to predict if a gene causes a disease. One of the features I have is a beta value (which ...
CommunityBot
- 1
0
votes
1
answer
70
views
Redundant feature after one hot encoding
I have a numerical feature called $x$ and a categorical feature called $y$.
$y$ is an ordinal feature (A,B,C,D,E,F).
I am using label encoding for my y feature and when I am seeing the correlation ...
CommunityBot
- 1
1
vote
1
answer
281
views
Rapidminer and decision tree weights
In Rapidminer, are the decision tree's weights a measure of the "importance" of attributes in the splitting procedure ?
If yes, why is useful to know these weights ?
Are there better methods to know ...
CommunityBot
- 1
2
votes
1
answer
186
views
How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?
I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
CommunityBot
- 1
2
votes
1
answer
2k
views
Variance threshold with python problem
i’am a beginner in scikit-learn and i’ve a little problem when using feature selection module VarianceThreshold, the problem is when i set the variance ...
CommunityBot
- 1
1
vote
3
answers
93
views
How can we shorten our questionnaire to only ask the most informative question at each point?
Our product has an onboarding questionnaire which asks the same 58 questions (with numeric answers) to every new user. That’s a lot of questions, so we’d love to reduce the number of questions we ask ...
CommunityBot
- 1
1
vote
1
answer
106
views
Queries regarding feature importance for categorical features
Queries regarding feature importance for categorical features:
Context:
I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
CommunityBot
- 1
1
vote
1
answer
83
views
How should I engineer features for Named Entity Identification task?
I was working on Named Entity Identification (not recognition) task. In this NLP task, given a sentence, model has to predict whether each word (aka token) is named entity or not. The dataset used ...
CommunityBot
- 1
1
vote
1
answer
25
views
Feature selection for propensity model
I'm trying to build a propensity model for whether or not a customer will buy a second product. I was given data that looks like this:
| Age | Income | DaysSince1stPurchase | Bought2ndProduct |
|:---- ...
- 626
0
votes
1
answer
16
views
How to represent facial features from video and classify high/low personality traits from facial features?
The dataset has 3-minute 30fps video conversations (no audio) of 150 extroverted and 150 introverted individuals. The goal is to classify them as "introverts" or "extroverts" based ...
- 556
1
vote
1
answer
204
views
how to test if labels have actual dependencies on features?
I am trying to train an LSTM(many to one) model with multivariate time series input and a categorical output.
after training for quite some time, the resulting model still has low accuracy and high ...
CommunityBot
- 1
0
votes
0
answers
6
views
Simultaneous Feature Selection and Time Series Selection
I have about 2000 features and 14 times series for them to predict. I am trying to reduce my feature count, but also my time series count. The goal is to find the 30 core features best at predicting ...
- 1
1
vote
1
answer
90
views
Brute-force feature selection and cross-validation
There is an existing score made of 10 parameters; each parameter is equally weighted & the total score is found by summing the score for each parameter.
I want to try to reduce the number of ...
CommunityBot
- 1
2
votes
1
answer
80
views
Using cross validation score to perform feature selection
So to perform my feature selection I ran cross validation over and over again, each time trying different subsets of my attributes and repeated this until I got the best cross validation score I could ...
CommunityBot
- 1
2
votes
1
answer
135
views
scikit-learn OMP mem error
I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error.
The machine has ...
CommunityBot
- 1
0
votes
1
answer
121
views
How to use scikit-learn to extract features from text when I only have positive and unlabeled data?
I'm looking for something similar to this
https://scikit-learn.org/stable/auto_examples/text/plot_document_classification_20newsgroups.html#sphx-glr-auto-examples-text-plot-document-classification-...
CommunityBot
- 1
2
votes
1
answer
190
views
Feature Selection on Aggregated Targetdata
I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
CommunityBot
- 1
0
votes
0
answers
19
views
Why is the feature direction chosen in the direction associated with largest eigenvalue of $Σ_T$ in case of more than two classes?
Why is the feature direction chosen in the direction associated with largest eigenvalue of $Σ_T$ in case of more than two classes? Please see the following.
- 101
1
vote
4
answers
7k
views
Feature importance in neural networks
Hello I am using keras to develop a neural network model and I have a data of 45 numerical predictor variables, 2 categorical targets that will be predicted each with a different model. As I found, ...
- 1