Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

154 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
63 views

Features reduction for the not correlated data set

I am working with classification problem on a training data set, which have 100 features. All the features in pairs haven't visible correlation. One can see it in the example pair plot for the some of ...
4
votes
0answers
56 views

Model for Differing Number of Rows per Observation

Looking to build a response model (click or no click) on marketing data which displays varying number of offers to a person. I don't want to model which offer they click but do they click any of the ...
3
votes
0answers
29 views

Features selection with a lot of dummy variables in R

I am performing features selection on 3849 dummy variable (one-hot encoding) using Boruta algorithm and the algorithm is taking forever to run. Is there a faster way I can perform features selection ...
3
votes
1answer
76 views

Feature selection for time series prediction

I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables. When calculating correlation between variables (e.g. Close price of Tesla vs ...
3
votes
0answers
2k views

Isolation Forest Feature Importance

As of scikit-learn version 0.19.1, there is no implementation for calculating feature importance in an Isolation Forest. I'm also having trouble finding any online resources proposing ways to get at ...
3
votes
0answers
55 views

Determine the most important documents for supervised learning

I have somewhat of a general/high level question. Assume I'm doing supervised machine learning on some text data (tweets for example) and categorizing the documents to a certain taxonomy (multi-class ...
2
votes
0answers
15 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...
2
votes
0answers
185 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
2
votes
1answer
136 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
2
votes
0answers
57 views

Structured Support Vector Machine (Joint Feature Map)

I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine) The theory's clear, but I need a tangible example to make everything more concrete. ...
2
votes
0answers
453 views

Combine multiple features for text classification

Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I'm trying to make my own classification algorithm (the text sends a positive/...
2
votes
1answer
27 views

Classifying objects based of a varying number of the same type of feature vector for each object

For a congressional session, I have created a doc2vec model of speeches made. Using the vectors from this model, I have a dataset of each congressperson, their political affiliation, and a list of the ...
2
votes
0answers
28 views

Reduce number of vectors in dataset to achieve the “same average dimensions result”?

I have many tests (rows), each with a large set of 3D vectors (features/cols). Each vector complies: Xn + Yn + Zn = 1 Simply averaging all components ...
2
votes
0answers
87 views

Feature Importance Python

My dataset has around 1000 features and 30k rows. All the feautres have value either 1 or 0. My target variable is Size which 3 classes : Small, Medium and Large. I have around 5k "small" data ...
2
votes
0answers
245 views

Metrics to evaluate features' importance in classification problem (with random forest)

I want to evaluate the importance of each of the features of a 2000x60 dataset in a classification problem with random forest. The most widely used ones apparrently are: Cross Entropy-Information ...
2
votes
0answers
89 views

Representing a community as a vector

My setup is this: Suppose I have transactional data over a large period of time. The parties of each transaction are labled, and I use Louvain algorithm for detecting communities (and sub-communities)...
2
votes
1answer
38 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
2
votes
0answers
90 views

How to do feature engineering for email cleaning / text extraction?

I have a large batch of email data that I want to analyse. In order to do that, I need to first prepare the data, as the messages are quite often >80% noise. Generally speaking, my dataset's structure ...
2
votes
0answers
170 views

Skewed distributions in predictive models

What are the issues of dealing with highly skewed variable in a supervised problem? What are the machine learning algorithms that suffer more from skewness in the data and what are the solutions to ...
2
votes
0answers
141 views

Giving Emails as Input to Machine Learning Algorithms

I want to classify emails as Spam and Non-Spam. I have a labelled dataset of 20,000 emails in TXT format. The emails are in individual files and also in one combined file. An example email looks ...
2
votes
0answers
1k views

Feature extraction using autoencoder and assigning sub-features to the classes

I have a dataset with N records and D numerical attributes belonign to C different classes. ...
2
votes
0answers
44 views

Am I correct in finding correlations

I want to perform feature selection, having 128 real-valued standardized features and 1/0 labels. Below are feature a5 density histograms for Classes 1 and 0. The data is skewed, so that Class 1 is ...
2
votes
0answers
403 views

randomForest::varImp VS conditional variable importance

Data: My training set consists of ~450k obs and 26 variables, out of which 1 is an ordinal factor (order_month, 12 levels) and the rest is numerical. Moreover, some of my predictors are highly ...
2
votes
0answers
111 views

What's the strategy for deciding which feature level is excluded from one hot encoding of a categorical variable?

I'm working on a regression problem with a continuous dependent variable (sale price of a home). Amongst my features are several categorical features, which I'm transforming to "one hot encoded" dummy ...
1
vote
1answer
39 views

How can we convert time series data to supervised learning problem?

I am preparing a data for machine learning model. I want to deal with time series data as normal supervised learning prediction. Let's say I have a data for car speed and I have several cars models ...
1
vote
0answers
28 views

Linear Regression on data with bimodal outcome

I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
1
vote
0answers
23 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
1
vote
0answers
21 views

Increase accuracy of occupancy prediction?

I have a project that's aimed to predict the amount of occupants at my local gym given the date and weather. Here's my Kaggle kernel I have two datasets, occupants on a given hour and weather on a ...
1
vote
1answer
31 views

Implicit feature selection

I have heard that Random Forest and other tree based machines apply some kind of implicit feature selection. My Question is: Does this also apply for machines like the SVM? As far as I understand is ...
1
vote
0answers
23 views

Difference between RFECV and SFS?

I used scikit.REFCV and mlxtend.SFS (backward) on the same data, same classifier, same cv, same scorer,... I also did a third version with sample weights passed to SFS's estimator And i'm conflicted ...
1
vote
0answers
22 views

Scikit-compatible network lasso implementation

Is anyone aware of a scikit-compatible network Lasso (nLasso) implementation? These papers have source code as well: D. Hallac, J. Leskovec, and S. Boyd, “Network lasso: Clustering and ...
1
vote
1answer
27 views

For a regression model, can you transform all your features to linear to make a better prediction?

I was thinking. Would it be a good approach to check your features one by one (assuming you have a manageable amount of them) and see the relationship they have with your target variable, if they ...
1
vote
1answer
30 views

Unable to understand which features to choose

I am a newbie here, but I am trying to work with a dataset which gives the attempt at the goal by a footballer,which will predict one of 2 possible outputs - whether or not they could score the goal ...
1
vote
1answer
48 views

How to use Random Forest to reduce dimensions

I am working on the Boston competition on Kaggle and at the moment I am trying to use Random Forest to find the columns with the highest correlation with the target variable ...
1
vote
1answer
20 views

ML Approach for Getting List of Observations with Similar Features (Discrete+Continuous)

I have a dataset with 19k observations. Each has approximately 448 features: - Text description turned into vectors of size 300 - 16 categorical variables represented numerically - The remainder ...
1
vote
0answers
46 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
1
vote
0answers
22 views

Orange 3 - How can a String feature behave as a coefficient?

I'm studying machine learning with data. I have a table including features and a target variable which is the price as in the following. When I want to figure out the coefficients to obtain linear ...
1
vote
0answers
25 views

if new feature downgrade the score for xgboost what do I have to look at?

let say I'm predicting the housing price of Boston(kaggle). if I got some score x then I added new feature y_K if this new feature drop the score. what is wrong with this feature and what do I ...
1
vote
1answer
135 views

Hierarchical Clustering and Variable Selection

I am using "Single linkage" hierarchical algorithm to cluster my data points with Gower Distance as my data have both qualitative and quantitative variables. After applying this for the full ...
1
vote
0answers
55 views

Feature selection with information gain (KL divergence) and mutual information yields different results

I'm comparing different techniques for feature selection / feature ranking. Two of the techniques under scrutiny are the mutual information (MI) and the information gain (IG) as used in decision trees,...
1
vote
0answers
116 views

Remove attributes with missing values exceeding a given threshold in WEKA

I imported csv file into WEKA, i have features that have missing value that has missing value percentage of 70% or above, i want to remove these features by weka or also sort that features by missing ...
1
vote
0answers
35 views

Is there a good online course for working with sklearn MLPClassifier?

I'am implementing an App for which I need a neural network. Because: I want to classify each DOM-text-element of an webpage which conains any curriculum vitae of a person. The neural network should be ...
1
vote
1answer
24 views

Feature selection/reduction techinique for combination of features in image processing

I have a combination of features extracted from 3 descriptors, namely GLCM based feaures(correlation, homogeneity,energy and contrast ), Local binary patterns (256) and discrete wavelet transform ...
1
vote
1answer
35 views

Feature Selection Phase

I am trying to predict the overall age of an opportunity (creation date - closing date) this is my response variable lets say an opportunity passes through 3 stages to close For example: Opp x ...
1
vote
0answers
14 views

How do I store/model data needed for my recommendation module?

I'm reading data from a store's product catalog, a 100mb xml file which contains product-wise attributes like prices, categories, etc. Given a product_id, my job ...
1
vote
1answer
93 views

Mapping between original feature space and an interpretable feature space

I'm reading the following really interesting paper https://arxiv.org/pdf/1602.04938.pdf on local interpretable model explanations on page 3 however particularly section 3.3 Sampling for Local ...
1
vote
0answers
174 views

Feature Selection: Having a large number of features select those features that classify the target class best

I have time series data and I'm using the cesium package to extract several features. Now the thing is, the number of extracted features is pretty significant. I have tried various visualizations to ...
1
vote
0answers
114 views

Perform Pearson's correlation and chi-squared test for feature selection in a dataset with a mixed type of features

I've a dataset of about 200 features and 5000 instances. These features comprise of different data types like percent (string like 4.50%), dollar amount (value between $0 - $1,000,000), discrete ...
1
vote
0answers
19 views

Does it exist feature selection/reduction techniques in $O(n \cdot d)$?

I'm curious to know if feature selection and/or feature reduction techniques exist, which are linear on number of data $n$ and on number of dimensions $d$. References and source code are very welcome....
1
vote
0answers
23 views

Suggestions on using model in production 1 test at a time

I have created an Artificial Neural Network with 4 categorical features and a binary outcome either 1 for suspicious or 0 for non-suspicious: ...