Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
675
questions
0
votes
0answers
7 views
Can I apply Feature Selection even though the number of features is smaller than the number of observations?
I was reading An Introduction to Statistical Learning when I came across the High Dimensional section, they argue that high-dimensional data causes a lot of problems. My question is, it is necesary to ...
0
votes
1answer
19 views
Why might my validation loss flatten out while my training loss continues to decrease?
In my effort to learn a bit more about data science I scraped some labeled data from the web and am trying to classify examples into one of three classes. I am running into a problem that, regardless ...
0
votes
1answer
13 views
How can I compute the ideal variance threshold value for my data?
I have a dataset that contains n features scaled between [0,1]. I would use an unsupervised feature selection algorithm (variance thresholding). How can I compute the threshold value?
0
votes
0answers
10 views
Determine most important features per class in binary classification?
The only good answer i found was this :
https://stackoverflow.com/questions/33118361/determine-most-important-feature-per-class
But the above answer doesn't work in binary classification because ...
0
votes
0answers
11 views
Preprocessing and feature selection in group k fold
I have experimental data collected from 10 people. From each person, 100 data points were collected under condition A, and 100 data points were collected under condition B. So, in total I have 10*(100+...
3
votes
2answers
40 views
Different feature importance results between DNN, Random Forests and Gradient Boosted Decision Trees
I've been modeling metabolite data with 3 different regressor models. I get similar results from running feature importance with Random Forest model and Gradient Boosted Decision Trees (where I used ...
2
votes
1answer
76 views
From logistic regression to XGBoost - selecting features to run the model with
I have been asked to look at XGBoost (as implemented in R, and with a maximum of around 50 features) as an alternative to an already existing but not developed by me logistic regression model created ...
0
votes
0answers
13 views
Which is the best way to select categorical features with Autoencoders in Python?
I have a dataset containing both categorical and numerical features. I am trying to work with Autoencoders for feature selection, so the first thing I do is to normalise the numerical features. For ...
2
votes
2answers
41 views
Repeated features in Neural Networks with tabular data
When using algorithms like linear regression or least-squares methods, having repeated or highly correlated features can be harmful for the model. For tree based models, they are generally not too ...
0
votes
0answers
13 views
How to correctly label images with multiple objects
I have 3 types of images:
A: Images of apples
B: Images of bananas, however some of these banana images also contain apples in the observable background
Is it enough to just label the bounding boxes ...
1
vote
0answers
16 views
Can convolutional network learn structural properties of one feature w.r.t to other?
I'm going through the literature on pose-estimation ( DeeperCut, OpenPose, MultiPersonPosetrack).
I'm interested in knowing whether these networks/ generally a CNN can learn properties (geometrical) ...
0
votes
1answer
16 views
Is the result of feature extraction a feature representation?
If a use a feature extraction method on images, do I then get a feature representation or is there a different meaning behind feature representation?
To my understanding, when I use a CNN on an image ...
0
votes
0answers
19 views
Noisy features detection
Based on the library featexp I am trying to identify noisy categorical features. I want to know if this is the right way and if there is some library for this solution it will be great to know it.
I ...
2
votes
0answers
41 views
Can one perform Feature Selection on a subset of training data?
I have a training data set with almost one million rows and I am considering eight features initially. My machine learning model will be Random Forest regressor.
In Section 3.4.7 of "Feature ...
1
vote
1answer
32 views
SelectFromModel vs RFE - huge difference in model performance
Note: I have already looked at Difference between RFE and SelectFromModel in Scikit-Learn post and my query is differnt from that post
Expectation:
SelectFromModel ...
0
votes
0answers
21 views
Feature Selection before modeling with Boosting Trees
I have read in some papers that the subset of features chosen for a boosting tree algorithm will make a big difference on the performanceso I've been trying RFE, Boruta, Clustering variables, ...
3
votes
2answers
61 views
How do you, analytically, show you are not using too many features?
One of the managers at my company asked if there is a I could include a metric demonstrates that the my model is not using too many Features/Variables. Is there a metric or best practice that does ...
1
vote
0answers
25 views
How to balance time/effort with transformations, feature selection, and models efficacy in nlp? [closed]
Edit: Question has been edited for reopening (see comment section for justification)
Being to new text analytics, I haven't gotten the hang of navigating a typical workflow given the longer times ...
-1
votes
1answer
17 views
How to choose variables to perform Exploratory Data Analysis
I have a dataset with about 110 variables. I have a target variable and I want to do an exploratory data analysis to find out what factors affect this target variable.
In such scenarios when we have a ...
1
vote
1answer
28 views
Can we use feature selection and dimensionality reduction together?
I have a dataset having about 10,000s of features. The features have a hierarchy inherent to them. I found an algorithm performing feature engineering, taking the hierarchy of the features into ...
0
votes
1answer
19 views
Terminology in machine learning: exogenous features vs external features
I am currently writing a scientific paper and do not know whether to call some of my input features of my neural network either external or exogenous.
My neural network receives as input features like ...
2
votes
1answer
62 views
Scikit-learn SelectKBest is picking up obviously unwanted Features
Dataset
Dataset Summary: Bank Loan (classification) problem
Problem Summary:
I am exploring ways to simplify EDA Process (Exploratory Data Analysis) of finding the best fit variables
I came across ...
1
vote
1answer
29 views
How to find the feature regions where each label is the most expected when using decision trees?
Given a decision tree for classification for example this one:
What is the way to find the feature domain (petal and sepal width and length) where a sample would most likely occur in the feature ...
1
vote
0answers
30 views
Handling highly correlated features [closed]
I have a data set of transactions and want to build a fraud detection model (classifier). Only 3 variables are given that could be used as input features. The number of transactions during past 3, 6 ...
1
vote
1answer
29 views
How to handle a valuable feature that is missing on 99\% of the samples in the data set?
Suppose we have an input feature that is highly predictive of the outcome we want to predict. However, the feature is missing on 99% of the samples in the data set. What is the best way to use this ...
1
vote
1answer
21 views
Drastic shift in feature importance upon adding other features
I have a model (GBDT) where adding a feature X is not important (according to SHAP), but when I add other features, and add X again, now feature X is the second most important!
What could explain that?...
1
vote
0answers
15 views
Justification for keeping features that do not provide separation using Random Forest
I have a random forest classifier model with approximately 70% accuracy; when I remove some variables that allow less separaion, I remain with the exactly same accuracy. However, I did not test this ...
3
votes
0answers
73 views
Best way to remove useless features when there are more than 100,000 features?
I am in a situation where i have more than 100,000 features, and i need to select the top features to give them to my final neural network model.
So far i have been using RandomForestClassifier in ...
1
vote
1answer
42 views
Is Recursive Feature Elimination finding best features subset?
On a set of 9 features I have applied Recursive Feature Elimination (RFE) algorithm using SVM estimator, following approach from (1). When requesting a subset of size 1 to be found, then RFE returned ...
2
votes
1answer
24 views
Feature Selection - Conditional Entropy
I've developed an algorithm to define conditional entropy for feature selection in text classification. I'm following the formula at Machine Learning from Text by Charu C. Aggarwal (5.2.2). The author ...
1
vote
1answer
39 views
Why we can't Remove features with missing values in Data Preprocessing [closed]
In a Real Time Dataset, There are many missing values available in the Dataset and also we need to deal with data preprocessing. And there are many ways to minimize the problem of missing values ...
1
vote
1answer
15 views
How we can Identify Specific Feature from a larger amount of Dataset?
In Machine Learning, we need to play with any kind of datasets.
In the Dataset, There are too many records and features, Some datasets had lots of features (sometimes it's called ...
1
vote
1answer
35 views
How to apply multiple filter in Data Frame? [closed]
How to implement multiple filters for checking data cell in a range ?
Suppose, I have a list of numbers like,
...
1
vote
1answer
20 views
Marginal contribution of a text document
I'm trying to build a Shapley value (marginal contribution) of a text document in terms of information content, given that there are several documents on a given topic.
For example, we have 3 reports ...
0
votes
0answers
9 views
Pytorch DNN feature importance / mean decrese accuracy
I have been working on a classification DNN for metabolite data. I would like to implement a feature importance calculation to the DNN, and I saw a paper that used mean decrease accuracy for this. ...
1
vote
0answers
34 views
How to Fit S shape (Sigmoid Function) in my scatterplot [closed]
How to interpret my chart? I want to get the maximum likelihood in logistic regression with this result (I'm really not sure if this is how it looks like):
I am currently using logistic regression to ...
0
votes
2answers
24 views
Recommended Tutorial Videos or Books on Feature Engineering Using Python [duplicate]
I will appreciate it if you guys can recommend for me a good hands-on tutorial videos or books on feature engineering using Python. I do not want videos or books that teach only the theory behind ...
0
votes
0answers
30 views
Can autoencoder latent variables to be used as features for classification?
I did some experiments on convolutional autoencoder by increasing the size of latent variables from 64 to 128. I used 4 covolutional layers for the encoder and 4 transposed convolutional layers as the ...
9
votes
2answers
152 views
Does “feature importance” depend on the model type?
I was working on a small classification problem (breast cancer data set from sklearn), and trying to decide which features were most important to predict the labels. I understand that there are ...
0
votes
1answer
52 views
Shall I use ordinal encoding or One-Hot-Encoding when using DBSCAN for content clustering on websites?
I want to cluster the preparation steps on cooking recipes websites in one cluster so I can distinguish them from the rest of the website.
To achieve this I extracted for each text node of the website ...
1
vote
0answers
54 views
The Merits of Feature Reduction Routines
I am interested in learning what routine others use (if any) for Feature Reduction/Selection.
For example, If my data has several thousand features, I typically try {2,3,4} things right away depending ...
3
votes
2answers
58 views
Treating missing data in categorical features
I have a dataset with one of the categorical columns having a considerable number of missing values. The interesting thing about this column is that it has values only for a particular category in &...
0
votes
0answers
23 views
Feature selection by involving validation dataset
I need expert advice about a small algorithm created to perform features selection. I have used a genetic algorithm to perform features selection based on a specific objective function (good accuracy &...
1
vote
0answers
22 views
Is there a common relationship between data inputs and the number of attainable features?
Is there a known relationship between the amount of information gain that comes from new data added to a dataset?
for eg: If I have a plant watering system that tells me:
An integer of how wet the ...
1
vote
0answers
25 views
Is there any feature selection method specific for regression analysis?
Is there any feature selection method that works especially well for regressions?
I used backwards elimination and forward selection before a lot but I've recently read that even though it's ...
0
votes
0answers
22 views
Is there a certain threshold over which to accept or reject predictors based on correlation values with the target variable?
I have been working on the Titanic dataset. After some feature manipulation, I printed out the correlation values between my target variable Survived and all the ...
0
votes
2answers
53 views
Regression performance with Feature Selection
I would like to ask you a theoretical question. In my project I am trying to get a better performance from my regression model by feature selection methods, especially with CatBoost feature ...
0
votes
0answers
8 views
Are there labeled multivariate time series data sets where only a subset/partial number of time series is relevant for each class?
I am looking for a data set of multivariate time series data which contains classes that only require a subset of time series for their identification. Assume a human activity recognition data set ...
2
votes
1answer
60 views
Feature Selection and Outlier Detection
How does feature selection impact outlier detection and also, removing outliers impact feature selection?
It could be a basic question. However, just to know the boundaries, I asked. Thanks in advance....
1
vote
0answers
23 views
Choosing the best set of features when forced to choose M out of N available features
Given: N features that map to some label Y using a Neural Net(NN)- it's a classification problem.
Problem: I want to get away by using only a subset of features denoted by M, where M<N.
Now I am ...
