Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

187 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
1answer
67 views

Model for Differing Number of Rows per Observation

Looking to build a response model (click or no click) on marketing data which displays varying number of offers to a person. I don't want to model which offer they click but do they click any of the ...
4
votes
0answers
2k views

Isolation Forest Feature Importance

As of scikit-learn version 0.19.1, there is no implementation for calculating feature importance in an Isolation Forest. I'm also having trouble finding any online resources proposing ways to get at ...
3
votes
2answers
67 views

Decision Trees and Feature Selection

I'm trying to experiment with the performance of different machine learning algorithms before and after applying feature selection. I tested SVM, Random Forest, KNN, Linear Regression, and, Decision ...
3
votes
0answers
705 views

How to do feature selection for clustering and implement it in python?

I am trying to implement k-means clustering on 60-70 features and I came across a post for feature selection technique on quora by Julian Ramos, but I fail to understand few steps mentioned. I am ...
3
votes
0answers
326 views

Linear Regression on data with bimodal outcome

I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
3
votes
0answers
50 views

Features selection with a lot of dummy variables in R

I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
3
votes
1answer
533 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
3
votes
1answer
294 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
3
votes
0answers
122 views

Feature Importance Python

My dataset has around 1000 features and 30k rows. All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
3
votes
0answers
346 views

Metrics to evaluate features' importance in classification problem (with random forest)

I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest. The most widely used ones apparrently are: Cross Entropy-Information ...
3
votes
0answers
56 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
3
votes
0answers
196 views

Skewed distributions in predictive models

What are the issues of dealing with highly skewed variable in a supervised problem? What are the machine learning algorithms that suffer more from skewness in the data and what are the solutions to ...
2
votes
0answers
13 views

Adding high p-value and low R square features in linear regression model to improve result

I am working on a linear regression problem. The features for my analysis have been selected using p-values and domain knowledge. After selecting these features, the performance of $R^2$ and the $...
2
votes
1answer
115 views

How to interpret a specific feature importance?

Apologies for a very case specific question. I have a dataset of genes, with which I am using machine learning to predict if a gene causes a disease. One of the features I have is a beta value (which ...
2
votes
0answers
22 views

Which stage should the correlation analysis be done?

I was thinking about it, but I couldn't find a logical explanation. Mostly im following below steps after data become ready: Correlation analysis and elimination Apply dummy if categorical variables ...
2
votes
0answers
17 views

stacking features vs concatenating layers

I am trying to get to the logical intuition of differences between stacking multiple features and passing it via a final block (which could comprise multiple layers and lets say a final classification ...
2
votes
0answers
12 views

features to help distinguish between document images

we are trying to build a model to classify different types of documents as the first step in our pipeline (final goal is to read all the text). Currently we use ImageNet to extract the features and ...
2
votes
1answer
65 views

Logistic Regression Model for categorical features with multiple values in each category

I am working on an insurance use case to build a logistic regression classifier to predict if a policy will lapse or not. The dataset has more than 20 categorical features for a policy. Each ...
2
votes
1answer
48 views

Feature Selection on Aggregated Targetdata

I have a question about feature selection on a dataset where the target variable is aggregated by the sum of different data points. I want to predict the number of sales depending on a variety of ...
2
votes
0answers
36 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...
2
votes
0answers
77 views

Feature selection with information gain (KL divergence) and mutual information yields different results

I'm comparing different techniques for feature selection / feature ranking. Two of the techniques under scrutiny are the mutual information (MI) and the information gain (IG) as used in decision trees,...
2
votes
0answers
72 views

Structured Support Vector Machine (Joint Feature Map)

I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine) The theory's clear, but I need a tangible example to make everything more concrete. ...
2
votes
0answers
661 views

Combine multiple features for text classification

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I'm trying to make my own classification algorithm (the text sends a positive/...
2
votes
1answer
28 views

Classifying objects based of a varying number of the same type of feature vector for each object

For a congressional session, I have created a doc2vec model of speeches made. Using the vectors from this model, I have a dataset of each congressperson, their political affiliation, and a list of the ...
2
votes
0answers
31 views

Reduce number of vectors in dataset to achieve the “same average dimensions result”?

I have many tests (rows), each with a large set of 3D vectors (features/cols). Each vector complies: Xn + Yn + Zn = 1 Simply averaging all components ...
2
votes
0answers
90 views

Representing a community as a vector

My setup is this: Suppose I have transactional data over a large period of time. The parties of each transaction are labled, and I use Louvain algorithm for detecting communities (and sub-communities)...
2
votes
1answer
110 views

Feature selection for time series prediction

I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables. When calculating correlation between variables (e.g. Close price of Tesla vs ...
2
votes
0answers
107 views

How to do feature engineering for email cleaning / text extraction?

I have a large batch of email data that I want to analyse. In order to do that, I need to first prepare the data, as the messages are quite often >80% noise. Generally speaking, my dataset's structure ...
2
votes
0answers
168 views

Giving Emails as Input to Machine Learning Algorithms

I want to classify emails as Spam and Non-Spam. I have a labelled dataset of 20,000 emails in TXT format. The emails are in individual files and also in one combined file. An example email looks ...
2
votes
0answers
1k views

Feature extraction using autoencoder and assigning sub-features to the classes

I have a dataset with N records and D numerical attributes belonign to C different classes. ...
2
votes
0answers
44 views

Am I correct in finding correlations

I want to perform feature selection, having 128 real-valued standardized features and 1/0 labels. Below are feature a5 density histograms for Classes 1 and 0. The data is skewed, so that Class 1 is ...
2
votes
0answers
438 views

randomForest::varImp VS conditional variable importance

Data: My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
2
votes
0answers
114 views

What's the strategy for deciding which feature level is excluded from one hot encoding of a categorical variable?

I'm working on a regression problem with a continuous dependent variable (sale price of a home). Amongst my features are several categorical features, which I'm transforming to "one hot encoded" dummy ...
2
votes
0answers
111 views

scikit-learn OMP mem error

I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error. The machine has ...
2
votes
1answer
27 views

is it a good idea to take the derivative or integral of some features and add them as new features in machine learning?

I'm learning how to do feature Engineering and come across some ideas in my head that's why I want to ask if I had some dataset with some features let's say 2 features and I have a timestamp column ...
1
vote
0answers
30 views

Literature on selecting specific dimensions in a word embedding vector

I am aware that the different dimensions in the word embedding represents different information and algebraic operations can be performed between two embeddings for example. Can anyone point me to ...
1
vote
0answers
8 views

How to deal with different audio formats for audio classification?

I am working on an audio classification problem statement to classify between two audio classes. I have collected samples from jotform, they are providing audio widget to collect .wav audio but it ...
1
vote
1answer
23 views

Can I apply feature selection before splitting by requiring selection occurs > 90% of time

I want to move the feature selection step to before splitting to save time and allow bigger input dataset. If, in repeated subsamples, a feature is selected in over X percentage of cases I will keep ...
1
vote
1answer
23 views

XGBoost - feature importance just depends on the location of the feature in the data

I'm trying to do some feature selection using XGBoost, but the feature importance chart just spits out the features in order of appearance. The feature that is in the first column in the xtrain data ...
1
vote
0answers
16 views

Random Forest feature selection

I am using Random Forest model for feature selection with data with bias. I have tried the Random Forest model on my features increasing number of trees and found features importance moving. Could ...
1
vote
0answers
18 views

Genetic algorithm - Feature selection packages in Python

Can you share some packages in Python which are implemented that I can use for selecting features based on a genetic algorithm? I did refer to this AUTO-ML post and found out that it is useful but ...
1
vote
1answer
16 views

How to interpret PCA rankings in Weka

I am struggling to understand what the rankings in Weka are representing. I.e. the coefficients for each attribute in the rank. What is the output in the Weka program for PCA telling me with these ...
1
vote
0answers
17 views

Feature engineering one step at a time or in bunches?

Currently, I'm working on my very first classification project. If you want to know what dataset I'm working with, think "playing stairway to heaven in your local guitar store", and it will probably ...
1
vote
1answer
17 views

In feature selection, I came across a situation where NaN were filled by median of the column values

Why the median value is used for NaN? Why not something else like mean? What is the logic behind using the median value?
1
vote
0answers
26 views

Why not use constant instead of permutation for model agnostic predictor importance?

I want to determine predictor importance. Ideal is to re-train same model on same dataset missing each variable in turn. This is too time consuming. The recommendation I have seen everywhere is to "...
1
vote
0answers
21 views

Does EDA helps only in case of linear regression?

I know what Explanatory data analysis is and how it helps us investigate and understand the data. What I dont understand is how does this help in case of nonlinear relationships? I mean if I'm using ...
1
vote
1answer
20 views

How to do backward features elimination when considering interactions between them

I have a multi linear regression problem, $Y$ is my target and $X_1, X_2, X_3$ are my features. In my regression, I consider the interaction between $X_1, X_2, X_3$ and I add a bias. So my problem ...
1
vote
0answers
13 views

What toolbox to use to create multi-output random forest(reggression) with custom spltting function at each node?

I am trying to implement "Real Time Head Pose Estimation fromConsumer Depth Cameras" by Fanelli et al. I need to train a random forest(regression) with the following criterion The predicted output is ...
1
vote
0answers
31 views

Model-independent measures for feature importance given highly correlated features

I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
1
vote
0answers
16 views

Do you need to perform variables reduction for tree-based models?

I know for methods and linear regression, GLM, Logistic regression, we typically run through a lot of variable reduction methods, i.e, forward/backward/stepwise, univariate analysis; variable ...