Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

SequentialFeatureSelector and scoring for clustering

I want to use SequentialFeatureSelector from scikit-learn to do feature selection for clustering (K-Means). [Here][1] is a list of available evaluation scores for clustering. What I do not understand ...
0 votes
1 answer
21 views

Feature engineering for interest-based age classification

I have a dataset which has users (rows) with the list of their interests (IABs), which looks like this ...
0 votes
2 answers
20 views

Using time serie to predict another variable

I would like to analyse head rotation data in space. For this I measured at 15HZ the rotation around the X, Y and Z angles for a little more than ten minutes. I would like to use these movements to ...
0 votes
0 answers
6 views

How is self-supervised learning different from feature selection or using dimensionality reduction

I am getting a bit confused in understanding self-supervised learning and how is it different from normal feature selection method. I understand that in self-supervised learning we are actually ...
0 votes
0 answers
21 views

Classification Algorithms for Geospatial Vector Data

I am new to the world of Geospatial Analysis. As such, I am interested in Machine Learning / Deep Learning techniques for classifying geospatial vector data specifically (as opposed to raster data). I'...
0 votes
0 answers
15 views

Should I use Boruta feature selection if I'm comparing two datasets?

I'm a beginner to machine learning and AI so I really could you some guidance on this. I am currently comparing two different datasets with different classification algorithms. One of the datasets has ...
0 votes
0 answers
15 views

Neural networks and input filters

In my use case scenario, I have a neural network that should filter the input and pass a specific value of the input array to the output. In particular, let's define the input as: ...
  • 1
0 votes
1 answer
48 views

Having problems replicating the results of a series of N LASSOs fit to N datasets in R

I have fit n LASSO Regressions on n different data sets (the 'datasets' object is an R list of length n where each element is a data.table which is a light and fast data frame from the data.table ...
  • 155
0 votes
1 answer
20 views

Can using the mean of absolute Shapely Values for feature importance give very wrong results?

In a classification problem, suppose a model has 2 variables, A and B, and the null model (the model without any variable) predicts 50% probability for belonging to class 1 for all the instances. Now ...
1 vote
1 answer
30 views

Why the marginal contribution of a feature is the difference between the feature effect minus the average effect

In several sources the marginal contribution is defined as the difference between the prediction with and without the feature. However, recently I read an article where the marginal contribution was ...
0 votes
1 answer
28 views

Best package and function in R to use to replicate my (Backward & Forward) Stepwise Regression results I got using step from the stat package

I am doing a research project as a 2nd author on a paper exploring the properties of a novel algorithm for Optimal Variable Selection where I am running the benchmark Variable Selection Methods. Each ...
  • 155
0 votes
0 answers
20 views

About feature importance in deep learning

For tree methods, I can plot the feature importance plot from tree.feature_importances_ in sklearn, is this achievable in deep learning (neural networks)? Is there ...
1 vote
0 answers
16 views

How recursive feature elimination with cross validation internally works?

I am trying to understand how recursive feature elimination with cross validation works (the RFECV on sklearn). Lets say that we have 10 features, and we perform <...
  • 171
1 vote
1 answer
66 views

Does sklearn perform feature selection within cross validation?

I would like to add a feature selector on my pipeline and use gridsearchcv to tune both the hyperparameters of the selector and the classifier(s). I am wondering if sklearn performs feature selection ...
  • 171
1 vote
2 answers
46 views

Training a Model that Doesn't Always Have All the Features

I am creating a model that gathers data from multiple sources and determines a confidence level for an instance that is common across those sources (ie. all sources have different features, but all ...
0 votes
0 answers
22 views

How to actually begin using my first EC2 instance on AWS in order to run regressions too big for me to run locally

I am using the RStudio Server Amazon Machine Image (AMI) for a collaborative statistical learning research project intended to produce a paper for publication because its computational requirements ...
  • 155
0 votes
0 answers
11 views

Why the order of the fearures affects synapse LightGBM predictions?

I am using LighGBM Classifier and Regressor and it seems that the order of the features I am adding, affect the predictions of the model. Everytime I change the order, another result comes up and with ...
0 votes
1 answer
28 views

Which Feature Selection Techniques for NLP is this represent

I have a dataset that came from NLP for technical documents my dataset has 60,000 records There are 30,000 features in the dataset and the value is the number of repetitions that word/feature appeared ...
  • 517
2 votes
1 answer
95 views

Would adding Elastic Net as an additional Benchmark add any value when LASSO is already an included benchmark?

I am currently engaged in a research project with a collaborator in which he is proposing a novel learning algorithm for optimal variable selection, and exploring its computational, statistical, and ...
  • 155
0 votes
1 answer
61 views

Quantifying the performance of Stepwise Regression ran on Monte Carlo generated datasets & comparing them to your method of interest

The source data files and scripts referenced here and from whom lines of code are included here can be found in my GitHub Repository for this collaborative research project exploring the properties of ...
  • 155
1 vote
1 answer
42 views

A good 4th Benchmark method to compare the performance of a novel Variable Selection Algorithm being evaluated

I am collaborating on a research project with a respected econometrician as a graduate student (although only in an MS program, not PhD program mind you) exploring the properties and comparing the ...
  • 155
0 votes
1 answer
29 views

Determine how each element in a vector contributes to cosine similarity when compare with other vector

I have a vector that represents my object and does a job of calculating which object is similar to the other object by using cosine similarity. To create that vector, I've combined many features that ...
0 votes
0 answers
9 views

How to compensate for different feature sampling in decision trees

I have a dataset, on which I would like to use a decision tree, where some features are sampled much less frequently than others. I am concerned that they could lead to suboptimal feature selection in ...
  • 1
0 votes
0 answers
20 views

Hypothesis testing

Suppose I am building a machine learning model and I have 20 features. My objective is to understand if there is sufficient statistical evidence of a relationship between the independent features and ...
6 votes
1 answer
235 views

Logistic Regression Modeling & Interpretation [closed]

I'm building a logistic regression model to predict the credit risk of lending company customers. I'm using dataset from kaggle : https://www.kaggle.com/datasets/ranadeep/credit-risk-dataset/code ...
0 votes
1 answer
28 views

Redundant feature after one hot encoding

I have a numerical feature called $x$ and a categorical feature called $y$. $y$ is an ordinal feature (A,B,C,D,E,F). I am using label encoding for my y feature and when I am seeing the correlation ...
0 votes
0 answers
30 views

SNS Clustermap with correlation with target value

I am trying to replicate a graph similar to this. https://link.springer.com/article/10.1186/s12864-017-4353-7/figures/1 I am trying to identify if combination of 2 features (30 features, 400000 rows/...
0 votes
1 answer
33 views

HalvingGridSearchCV

Is there a way to get Feature importance from sklearn`s HalvingGridSearchCV? For example: Is there any way to access the feature importance? Please help me up. Thanks!
  • 1
0 votes
0 answers
14 views

What is the functioning of Rotation Forest when replacing PCA with Random Subset Feature Selection (RSFS)?

So guys, i've been lately reading about Rotation Forest algorithm and i learned that it uses PCA in order to reduce the feature space and it also uses a decision tree as base classifier. The authors ...
0 votes
0 answers
20 views

How to perform feature engineering on columns with multiple categories?

I want to perform feature engineering on a data that mostly contains textual data and lot of columns with multiple categories like Supervisor, location code, employee class, business unit, job title, ...
0 votes
0 answers
22 views

Machine learning with mixed variables dataset (numerical, categorical and embeddings)

I'm working on a machine learning project where I'm trying to predict the revenue of a movie. My dataset contains mixed data types. There are numerical features (rating, number of votes, release year,....
0 votes
0 answers
53 views

python Packages for correcting covariate shift

I am trying to build a regression model to predict customer revenue. However, I see that my model almost always performs well on training and not on testing data. While I am building a parsimonius ...
  • 2,449
0 votes
0 answers
15 views

How to decide which feature in DataFrame can be deleted (feature selection)?

I'm new to Machine Learning, and I'm working on dataset "Combined Cycle Power Plant over 6 years (2006-2011)", when the power plant was set to work with ...
  • 1
0 votes
1 answer
28 views

Should I use "Recency" as an predictor for churn if I want to catch churners early?

I want to build a customer churn prediction model that predicts probability of churn the next day and I'm looking for some features that might be important for the target variable which has outcomes ...
  • 103
0 votes
1 answer
30 views

Variable selection and NA

I have a very large dataset with a lot of NAs in the data. I want to perform an analysis and have to select the variables that are of most interest. I feel like I have to take 3 steps before I can ...
  • 13
0 votes
1 answer
54 views

what are the solutions for efficient feature selection in a very large feature space?

I have a classification dataset (50k observations and 10 features) on which I can't get a good result.. I want to try increasing the number of features.. I plan to automatically generate many feature ...
  • 103
0 votes
0 answers
12 views

Dealing with categorical columns with unbalanced value count

I'm doing some data processing and wondering what is the best practice for dealing with categorical columns that has a value counts plot looking like the below (these are one-hot-encoded at a later ...
0 votes
0 answers
7 views

Can/should a logical multi-class classification anomaly detection system be described as "unsupervised machine learning"?

I would like to ensure that my use of terminology is accurate. My question is: what terminology should I be using in this case? The system I am building assigns classes (-1, 0, +1) to observations ...
0 votes
0 answers
11 views

how to effectively select "locally" important variables in regression (like an imbalanced variable with a lot of zeros and few 1)

so I have situations where there are some variables which I know are relevant from a business perspective which might decrease variance on a part of the dataset. example: while estimating cars values, ...
  • 159
0 votes
0 answers
31 views

Feature Engineering/FS

I've a dataset which consists of 33 features. Out of which first 32 features are latitude & longitude positions for 16 machines at 16 different locations (16latitude & 16 longitude values). ...
1 vote
1 answer
26 views

Why does feature selection matter if your model has L1 regularization?

I've been tinkering around with boosted trees, and I saw that for common libraries there is a parameter you can set to determine L1 regularization. I doubled my original feature set to around 130 ...
-1 votes
1 answer
27 views

Machine learning feature selection

I have 40 Independent variables in dataframe. Out of these 40 IV's 6 IV's are as follows, X1 X2 X3 X4 X5 X6 Target 1 2 3 4 5 6 21 2 4 1 3 5 4 19 As seen in above table, target variable value is a ...
  • 1
0 votes
0 answers
35 views

Trying to find a Model for multiclass classification problem where you want two classes close together but far away from a third?

So I've got this problem where I want to find out where two classes are more a like ( features wise) than a third. So for example if I have three classes {A,B,C} I want to find out where class A and B ...
0 votes
1 answer
221 views

How does SelectFromModel from scikit-learn select features?

When I use XGBClassifier with SelectFromModel the algorithm always returns around five features regardless of the ...
  • 193
0 votes
0 answers
19 views

What would be a good method for detecting symbols in an image and finding those elsewhere?

I'm trying to build a system that can detect symbols from a legend image, then find and count those symbols in a diagram For example the legend would have symbols like this, with a label next to them ...
0 votes
0 answers
19 views

Which redundant feature should be use

I have two redundant features. A & B with 0.85 correlation. I know only one of them should be used to trained my model, but which feature should i use? A or B? Is there any method that can i use ...
0 votes
1 answer
23 views

In between data partition and feature selection which one need to perform 1st

I need to perform a feature selection on my dataset. My dataset is an imbalanced dataset where the class of interest is the minority class. Therefore, recall and F2 measures are two important metrics ...
  • 359
0 votes
0 answers
30 views

Feature Importance and Threshold Moving

Problem Type : Binary Classification Dataset : Imbalanced Current sklearn pipeline uses XGBoost model and involves moving threshold from 0.5 to a considerably higher value like 0.8 - 0.9. Is it viable ...
  • 1
0 votes
1 answer
36 views

Encoding Categorical feature with high cardinality - in my case IP adresses

I'm working on an intrusion detection project, I have many categorical features, for some I used label encoding since I don't have many possible values. But for IP addresses, it's a high cardinality ...
  • 11
0 votes
0 answers
38 views

Feature importance with 0 score

I have feature importance score that showed like this picture. Why some of my features have 0 feature importance score (default, job_admin, education, job_enterpreneur, job_management, job_others, ...

1
2 3 4 5
19