Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
1answer
11 views

How to interpret Variance Inflation Factor (VIF) results?

From various books and blog posts, I understood that the Variance Inflation Factor (VIF) is used to calculate collinearity. They say that VIF till 10 is good. But I have a question. As we can see in ...
2
votes
1answer
38 views

Using historical label as a feature in my ML model?

I am working on a predictive model to predict change in the price of an asset (up, down, no change). The labeling is based on the derivative of the price and is exponentially smoothed with an alpha of ...
3
votes
1answer
76 views

Feature selection for time series prediction

I'm working on an LSTM-based stock market forecasting problem and trying to figure out a way to select input variables. When calculating correlation between variables (e.g. Close price of Tesla vs ...
1
vote
1answer
61 views

Effect of adding extra unrelated features to linear perceptron

Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
1
vote
1answer
39 views

How can we convert time series data to supervised learning problem?

I am preparing a data for machine learning model. I want to deal with time series data as normal supervised learning prediction. Let's say I have a data for car speed and I have several cars models ...
0
votes
2answers
14 views

How to reduce the Root mean square error

I have dataset which describe "how many passenger arriving in some airport " and I would like to predict how many passenger arriving in monthly bases for next year. The features that I have is the ...
0
votes
1answer
25 views

How does $\chi^2$ feature selection work?

I can't find the information how $\chi^2$ are used to select numerical features for a model. ...
1
vote
1answer
55 views

Feature selection or Dimension reduction in unsupervised learning

I'm trying to do Embedded clustering using kmeans. This is customer data, so it involves a lot of sentences, so I'm using the universal sentence encoder before clustering. But I should be doing a ...
0
votes
1answer
288 views

Knowing Feature Importance from Sparse Matrix

I was working with a dataset which had a textual column as well as numerical columns, so I used tfidf for textual column and created a sparse matrix, similarly for the numerical features I created a ...
1
vote
2answers
2k views

What are Laplacian scores and how do they effect feature selection?

I was going through a paper related to feature selection wherein I constantly came across the term Laplacian Scores which I was not able to understand. Can anyone explain their importance in feature ...
0
votes
1answer
11 views

Feature Selection algorithm/library for CRF

I am using the Conditional Random Fields CRF suite scikit-learn wrapper algorithm. I have read on the literature various approaches for feature selection, but I cannot find any on that package or, ...
0
votes
1answer
14 views

What are the steps and correct order of the operations in Machine Learning? [from Getting data to optimising models]

I've followed lots of tutorials on Machine Learning but in each of these, they go for a different strategy so it's quite confusing for me. I want to Know that what are the operations involved and what ...
3
votes
2answers
44 views

Should features be correlated on uncorrelated for classification and regression (prediction)

I have seen researchers using pearson's correlation coefficient to find out the relevant features -- to keep the features that have a high correlation value with the target. The physical implication ...
0
votes
1answer
30 views

Variation in output of Logistic Regression when using SMOTE

I am working on a logistic regression case with an imbalance in the target variable. To fix this I am using SMOTE (Synthetic Minority Oversampling Technique), but each time I run my regression model, ...
0
votes
1answer
110 views

Difference between Information Gain and Mutual Information for feature selection

What is the difference between information gain and mutual information? At this point, I understand that information gain is calculated between a random variable and target class for classification ...
0
votes
0answers
18 views

Audio dataset preprocessing to perform cry detection

I am building a neural network to perform cry detection (i.e., binary classification of cry/non-cry situations) when capturing sound in a house environment. To do so, I performed the following steps: ...
14
votes
2answers
3k views

How to combine categorical and continuous input features for neural network training

Suppose we have two kinds of input features, categorical and continuous. The categorical data may be represented as one-hot code A, while the continuous data is just a vector B in N-dimension space. ...
2
votes
1answer
13 views

sklearn.feature_selection vs xgboost feature_importances?

sklearn.feature_selection vs xgboost feature_importances Can somebody explain in-detailed differences between sklearn.feature_selection and xgboost feature_importances? And how the algorithms work ...
2
votes
0answers
15 views

When does random forest feature importance fail?

I'm curious about the assumptions of random forest feature importance. In this paper, the author says that "We show that random forest variable importance measures are a sensible means for ...
1
vote
3answers
90 views

adding logic combinations of boolean features in classification

I want to build a classifier from a dataset of vectors that include exclusively boolean values. Is there any chances that my classifier might perform better if, previously to the learning, I add ...
0
votes
3answers
216 views

Random Forests Feature Selection on Time Series Data

I have a dataset with N features, each one with 500 instances in time. For example, let's say that I have the following: Features ...
1
vote
0answers
28 views

Linear Regression on data with bimodal outcome

I have a data set with 3,000 features and continuous dependent variables of time with 18,000 instances. The histogram of the dependent variables show that the they have a bimodal distribution. I am ...
0
votes
0answers
25 views

how to use sklearn without feature selection

I am trying to study the effect of using feature selection onmy text classification code . I want to make a rating without any feature selection, but sklearn use document frequency (df) by default ...
1
vote
1answer
35 views

Association between features

Given the anonymized dataset of features below, where: "code" is a categorical variable. "x1" and "x2" are continuous variables. "x3" and "x4" are extracted features. They are the mean values of ...
0
votes
1answer
48 views

Increase accuracy of classification problem [closed]

I am trying to build a classifier that predicts the compiler given some operations of assembly code. Here is the pandas dataframe: What I do is using a TfidfVectorizer and select the features that ...
0
votes
0answers
95 views

How to use additional variables that are not available in test set?

I have additional variables in my dataset that are somewhat correlated to the continuous target variable, but that are completely unavailable in the test set. So, I'm wondering how the best to use ...
0
votes
0answers
15 views

Can I merge levels or factors having equal mean in categorical variable

I compared levels of categorical variable by their respected mean, obtain from continuous response variable using pivot table. I found that some of the levels is having nearly equal mean e.g 'BrDale' ...
1
vote
0answers
23 views

Extracting Features for Graph transformation

Suppose I have a directed graph G (V,E) whose transformation is defined by a library of patterns. Each vertex is of particular type. The library of patterns contain subgraphs (g1,g2,g3 etc)and it's ...
0
votes
1answer
54 views

Building an efficient feature vector

I am building a classifier for malware analysis, which predicts if I have a malware by looking at the intructions of an assembly code, such as push, mov,... and predicting the optimization method. ...
0
votes
0answers
17 views

Information Gain & Gini Index for NLP

I know how Information Gain and Gini Index work in General. I have problem figuring out how to apply these techniques in NLP and text feature extraction. Can someone show me an example of how to ...
0
votes
1answer
226 views

How to select variables based on the mean correlation in a correlation matrix?

I have a set of independent variables and I am calculating the correlation matrix between them using the Pearson Correlation Coefficient in Python. A part of the matrix looks like this: From this ...
0
votes
1answer
26 views

Problem building a feature vector

I am trying build a classifier for malware analysis for which basing in the instructions of an assembly code, such as push, mov,... I want to predict the compiler, and in a second time the ...
0
votes
1answer
35 views

Find the most relevant columns for each single class in pandas

The following question (this one) did not help me. I have a big dataset, and I want to know which Columns are the most relevant for the Target Variable. I know that, in my case, for each class in the ...
1
vote
1answer
31 views

Implicit feature selection

I have heard that Random Forest and other tree based machines apply some kind of implicit feature selection. My Question is: Does this also apply for machines like the SVM? As far as I understand is ...
0
votes
2answers
34 views

How to deal with a feature that has lot of categorical values?

I know this question has been asked before and I have tried a few things but those things are not working as expected for my usecase. I have a 500 length feature vector. One of these features is a ...
2
votes
3answers
6k views

Determine useful features for machine learning model

I am working with a dataset with hundreds of features. I wish to create a simple machine learning model using 7-10 features from the original dataset. My question is this: What quantitative metrics ...
0
votes
1answer
18 views

Pearson correlation make disapear the target column

I have a dataframe with some numerical and categorical values. I want to do some feature selection to visualise a low-dimensional split in the dataset when the target variable is the grade. Yet, when ...
1
vote
2answers
37 views

Feature selection filter methods

I am confused about when to use which filter methods for feature selection. I tried to learn them through online resources and found methods like chi-square, variance threshold, F-test, Mutual ...
2
votes
1answer
136 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
0
votes
0answers
17 views

Find out which attribute of a movie causes the most variation in score

I'm tinkering around with a subset of the IMDb dataset, and have been thinking about what specific attributes of a movie impact its user rating. I am looking to survey what methods can be used to find ...
0
votes
0answers
10 views

Recursive feature elimination on train data or complete dataset and dummy encoding

I am using RFE with logistic regression. I will also be doing cross validation with RFE (RFECV in sklearn) to get the optimum number of features. I am not sure whether to use RFECV on just train ...
0
votes
2answers
3k views

Features selection in KNN

I have a naive question about using the K Nearest Neighbor algorithm: is feature selection more important in KNN than in other algorithms? If a particular feature is not predictive in a neural ...
0
votes
1answer
20 views

Multivariate Multilag Regression with one shot prediction using LSTM

I am working on a multivariate regression task using a LSTM and I am interested in one shot prediction of my target variable (which is the price of a commodity). For example, the first parameter I ...
0
votes
0answers
26 views

if I got feature importance of xgboost/LightGBM what is next?

If I have feature importances of different variables in a xgboost/LightGBM model, how do I use this information? Is it better to just use the top n features and retrain the model? Does the feature ...
0
votes
1answer
97 views

xgboost feature selection and feature importance

when you create the new feature for data analysis for linear regression, it is clear that the feature has to be linear with other features is better but for xgboost what is the guideline to make a ...
0
votes
1answer
42 views

What variance threshold to consider in feature selection?

Consider a numerical dataset with continuous variables, that has been scaled to end up with values in the [0,1] range. How can I compute a reasonable variance threshold for all the variables?
1
vote
1answer
19 views

My models performs better with the arbitrary random feature. How can I interpret this?

I am training 6 different classifiers 'Decision Tree', 'Random Forest', 'Logistic regression' and 'SVM' with different kernels. There are about 80 dependent variables including categorical and ...
0
votes
1answer
164 views

How do I right feature selection for DBSCAN?

I want to use DBSCAN to recognize any clusters within all text elements from the DOM tree of any webpage. For example all menu items shall be clustered separatey to all main content or footer elements....
0
votes
1answer
19 views

How to choose the features for an algorithm from the given attached screenshot?

How to choose the features from the given attached heat map & correlation factor for the classification algorithm? I have 6 different features i.e., ac233fc01403, ac233fc02eaa, ac233fc015f6, ...
1
vote
0answers
21 views

Increase accuracy of occupancy prediction?

I have a project that's aimed to predict the amount of occupants at my local gym given the date and weather. Here's my Kaggle kernel I have two datasets, occupants on a given hour and weather on a ...