Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
973
questions
1
vote
0
answers
29
views
How to implement Ant Lion Optimization (ALO) feature selection for KNN Classification Problems?
I have been assigned for a project related to text data classification, i have preprocess and vectorized the data with TF-IDF. For feature selection i am using pyMetahueristic library to implement ALO ...
0
votes
0
answers
27
views
RFECV giving same test scores for every feature size
I am trying to reduce the feature space of my training dataset down from the 16000 it currently is before engaging in Random Forest Classification.
Initially I removed all features in less that 5% of ...
0
votes
1
answer
11
views
How to handle using input feature (clicks) when it is used in target too?
I am trying to create a ranking model, where I am thinking about creating ground truth based on clicks by user. But at same time past clicks made by users seems like a vital input feature too. Any ...
0
votes
2
answers
84
views
Different Algorithms for 50-50 A/B Testing
We are running A/B tests on web app customers, given a customerId. Each customer will see different web-feature designs. Trying to prevent usage of Feature Flags as its not currently setup yet in our ...
1
vote
1
answer
31
views
Different patterns in dataset
Suppose some features in a dataset have a linear correlation with the target variable, some have a polynomial correlation, and for categorical features, the target values tend to distribute ...
0
votes
1
answer
161
views
Testing accuracy is higher than training accuracy
My testing accuracy is way higher than my training accuracy. I have used feature selection and split the data into training, validation and test sets.
...
2
votes
1
answer
801
views
If my target variable is binary, is it better to use Pearson's or Spearman's for my correlation vector?
I'm using a corr vector, combined with RFE, to perform feature selection.
I keep reading conflicting things online as to whether I should use Pearson's or Spearman's...
0
votes
0
answers
21
views
Chi-Sqare test and Continuous values
How does chi-square test work for continuous variables. I see that it is used in most papers to test dependencies between continuous explanatory variables and the target variable. Please how does this ...
0
votes
0
answers
12
views
Feature Engineering and Score Generation for a List of Features
I have a decision tree trained for a binary classification task. It uses a model score plus some other attributes as features. It performs well, majorly because the model score is a strong feature ...
0
votes
0
answers
32
views
Where do I find dataset for rectangular patch antenna?
I am doing a project in my college, and for that I need a dataset containing the length, width , height along with return loss for different frequency of operation of the rectangular patch antenna. ...
0
votes
1
answer
41
views
How to treat single column with both continuous and categorical data for ML model
I am working on financial data where I have a feature(column) with 90% values between 0-1000 (continuous) and 10% values as -1, -2 and -9. (default values)
Default value definition:
-1: data not ...
0
votes
1
answer
40
views
important feature selection using dimensionality reduction algorithms
I have a dataset having more than 25000 features. I did perform noise removal using the histogram approach, and this dataset gets reduced to more than 5000 features. There are two classes, healthy and ...
0
votes
1
answer
38
views
Feature engineering for interest-based age classification
I have a dataset which has users (rows) with the list of their interests (IABs), which looks like this
...
0
votes
2
answers
34
views
Using time serie to predict another variable
I would like to analyse head rotation data in space. For this I measured at 15HZ the rotation around the X, Y and Z angles for a little more than ten minutes. I would like to use these movements to ...
0
votes
0
answers
60
views
Classification Algorithms for Geospatial Vector Data
I am new to the world of Geospatial Analysis.
As such, I am interested in Machine Learning / Deep Learning techniques for classifying geospatial vector data specifically (as opposed to raster data). I'...
0
votes
1
answer
65
views
Having problems replicating the results of a series of N LASSOs fit to N datasets in R
I have fit n LASSO Regressions on n different data sets (the 'datasets' object is an R list of length n where each element is a data.table which is a light and fast data frame from the data.table ...
0
votes
1
answer
174
views
Can using the mean of absolute Shapely Values for feature importance give very wrong results?
In a classification problem, suppose a model has 2 variables, A and B, and the null model (the model without any variable) predicts 50% probability for belonging to class 1 for all the instances.
Now ...
1
vote
1
answer
213
views
Why the marginal contribution of a feature is the difference between the feature effect minus the average effect
In several sources the marginal contribution is defined as
the difference between the prediction with and without the feature.
However, recently I read an article where the marginal contribution was ...
0
votes
1
answer
185
views
Best package and function in R to use to replicate my (Backward & Forward) Stepwise Regression results I got using step from the stat package
I am doing a research project as a 2nd author on a paper exploring the properties of a novel algorithm for Optimal Variable Selection where I am running the benchmark Variable Selection Methods. Each ...
1
vote
0
answers
46
views
How recursive feature elimination with cross validation internally works?
I am trying to understand how recursive feature elimination with cross validation works (the RFECV on sklearn). Lets say that we have 10 features, and we perform <...
1
vote
1
answer
520
views
Does sklearn perform feature selection within cross validation?
I would like to add a feature selector on my pipeline and use gridsearchcv to tune both the hyperparameters of the selector and the classifier(s).
I am wondering if sklearn performs feature selection ...
1
vote
2
answers
284
views
Training a Model that Doesn't Always Have All the Features
I am creating a model that gathers data from multiple sources and determines a confidence level for an instance that is common across those sources (ie. all sources have different features, but all ...
0
votes
1
answer
68
views
Why the order of the fearures affects synapse LightGBM predictions?
I am using LighGBM Classifier and Regressor and it seems that the order of the features I am adding, affect the predictions of the model. Everytime I change the order, another result comes up and with ...
0
votes
1
answer
49
views
Which Feature Selection Techniques for NLP is this represent
I have a dataset that came from NLP for technical documents
my dataset has 60,000 records
There are 30,000 features in the dataset
and the value is the number of repetitions that word/feature appeared
...
2
votes
1
answer
99
views
Would adding Elastic Net as an additional Benchmark add any value when LASSO is already an included benchmark?
I am currently engaged in a research project with a collaborator in which he is proposing a novel learning algorithm for optimal variable selection, and exploring its computational, statistical, and ...
0
votes
1
answer
75
views
Quantifying the performance of Stepwise Regression ran on Monte Carlo generated datasets & comparing them to your method of interest
The source data files and scripts referenced here and from whom lines of code are included here can be found in my GitHub Repository for this collaborative research project exploring the properties of ...
1
vote
1
answer
46
views
A good 4th Benchmark method to compare the performance of a novel Variable Selection Algorithm being evaluated
I am collaborating on a research project with a respected econometrician as a graduate student (although only in an MS program, not PhD program mind you) exploring the properties and comparing the ...
0
votes
1
answer
233
views
Determine how each element in a vector contributes to cosine similarity when compare with other vector
I have a vector that represents my object and does a job of calculating which object is similar to the other object by using cosine similarity.
To create that vector, I've combined many features that ...
6
votes
1
answer
258
views
Logistic Regression Modeling & Interpretation [closed]
I'm building a logistic regression model to predict the credit risk of lending company customers.
I'm using dataset from kaggle : https://www.kaggle.com/datasets/ranadeep/credit-risk-dataset/code
...
0
votes
1
answer
78
views
Redundant feature after one hot encoding
I have a numerical feature called $x$ and a categorical feature called $y$.
$y$ is an ordinal feature (A,B,C,D,E,F).
I am using label encoding for my y feature and when I am seeing the correlation ...
0
votes
1
answer
96
views
HalvingGridSearchCV
Is there a way to get Feature importance from sklearn`s HalvingGridSearchCV?
For example:
Is there any way to access the feature importance? Please help me up. Thanks!
1
vote
0
answers
78
views
Machine learning with mixed variables dataset (numerical, categorical and embeddings)
I'm working on a machine learning project where I'm trying to predict the revenue of a movie.
My dataset contains mixed data types. There are numerical features (rating, number of votes, release year,....
0
votes
1
answer
100
views
Should I use "Recency" as an predictor for churn if I want to catch churners early?
I want to build a customer churn prediction model that predicts probability of churn the next day and I'm looking for some features that might be important for the target variable which has outcomes ...
0
votes
1
answer
61
views
Variable selection and NA
I have a very large dataset with a lot of NAs in the data. I want to perform an analysis and have to select the variables that are of most interest.
I feel like I have to take 3 steps before I can ...
0
votes
1
answer
106
views
what are the solutions for efficient feature selection in a very large feature space?
I have a classification dataset (50k observations and 10 features) on which I can't get a good result..
I want to try increasing the number of features..
I plan to automatically generate many feature ...
0
votes
0
answers
28
views
Dealing with categorical columns with unbalanced value count
I'm doing some data processing and wondering what is the best practice for dealing with categorical columns that has a value counts plot looking like the below (these are one-hot-encoded at a later ...
0
votes
0
answers
18
views
Can/should a logical multi-class classification anomaly detection system be described as "unsupervised machine learning"?
I would like to ensure that my use of terminology is accurate. My question is: what terminology should I be using in this case?
The system I am building assigns classes (-1, 0, +1) to observations ...
0
votes
0
answers
42
views
Feature Engineering/FS
I've a dataset which consists of 33 features.
Out of which first 32 features are latitude & longitude positions for 16 machines at 16 different locations (16latitude & 16 longitude values).
...
1
vote
1
answer
73
views
Why does feature selection matter if your model has L1 regularization?
I've been tinkering around with boosted trees, and I saw that for common libraries there is a parameter you can set to determine L1 regularization. I doubled my original feature set to around 130 ...
-1
votes
1
answer
29
views
Machine learning feature selection
I have 40 Independent variables in dataframe.
Out of these 40 IV's 6 IV's are as follows,
X1
X2
X3
X4
X5
X6
Target
1
2
3
4
5
6
21
2
4
1
3
5
4
19
As seen in above table, target variable value is a ...
0
votes
0
answers
43
views
Trying to find a Model for multiclass classification problem where you want two classes close together but far away from a third?
So I've got this problem where I want to find out where two classes are more a like ( features wise) than a third. So for example if I have three classes {A,B,C} I want to find out where class A and B ...
0
votes
1
answer
511
views
How does SelectFromModel from scikit-learn select features?
When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the ...
0
votes
0
answers
234
views
What would be a good method for detecting symbols in an image and finding those elsewhere?
I'm trying to build a system that can detect symbols from a legend image, then find and count those symbols in a diagram
For example the legend would have symbols like this, with a label next to them ...
0
votes
0
answers
25
views
Which redundant feature should be use
I have two redundant features. A & B with 0.85 correlation. I know only one of them should be used to trained my model, but which feature should i use? A or B? Is there any method that can i use ...
0
votes
1
answer
46
views
In between data partition and feature selection which one need to perform 1st
I need to perform a feature selection on my dataset. My dataset is an imbalanced dataset where the class of interest is the minority class. Therefore, recall and F2 measures are two important metrics ...
0
votes
0
answers
51
views
Feature Importance and Threshold Moving
Problem Type : Binary Classification
Dataset : Imbalanced
Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9.
Is it viable ...
0
votes
1
answer
51
views
Encoding Categorical feature with high cardinality - in my case IP adresses
I'm working on an intrusion detection project, I have many categorical features, for some I used label encoding since I don't have many possible values. But for IP addresses, it's a high cardinality ...
0
votes
0
answers
234
views
Feature importance with 0 score
I have feature importance score that showed like this picture. Why some of my features have 0 feature importance score (default, job_admin, education, job_enterpreneur, job_management, job_others, ...
0
votes
1
answer
101
views
Select the best feature selection method for classification
I am trying to make predictions (using Weka) on a tabular dataset. It is a categorical dataset which is encoded by label encoder...
2
votes
2
answers
638
views
Correlation vs Mutual Information vs let-the-model-decide
I recently encountered the Mutual Information concept, and started reading on it. As I saw that it can get non-linear relations, it seemed to me that it might be a more powerful method to choose which ...