Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
2
votes
1answer
69 views

How to understand ANOVA-F for feature selection in Python. Sklearn SelectKBest with f_classif

I am trying to understand what it really means to calculate an ANOVA F value for feature selection for a binary classification problem. As I understand from the calculation of ANOVA from basic ...
0
votes
1answer
21 views

How to group categorical columns into similar types?

(Forgive me if the question is ill put. I am a novice in data science. Please comment or edit so that the question can be improved) I have a dataset where we have to predict the future sale of a shop....
1
vote
2answers
21 views

Decision tree with multiple outputs

I have a sample with 10 independent variables (X1, X2, X3 ....), and multiple output labels (y1, y2, y3). Here y1 will depend on X1, X2 y2 will depend on X3, X4 and so on. y1, y2, y3 might or might ...
1
vote
1answer
17 views

How to interpret PCA rankings in Weka

I am struggling to understand what the rankings in Weka are representing. I.e. the coefficients for each attribute in the rank. What is the output in the Weka program for PCA telling me with these ...
0
votes
0answers
24 views

Feature Selection with non-linear numerical and categorical variables

I have a dataset of 45 non-linear numerical values and 2 categorical values. I am making a feature selection to predict categorical variables one by one or together. I used the correlation ratio and ...
0
votes
0answers
5 views

Feature Selection Using Linear Discriminant Analysis

In this post it says that when input variables are continuous and response is categorical, in that case we can use Linear Discriminant Analysis (LDA). But as far as i know it is a dimentionality ...
1
vote
1answer
9 views

How and When features are attached to target label

I am using Mallet CRF library and having training set sequences like below. ...
0
votes
1answer
22 views

Should we do Feature selection in parallel with feature engineering?

I'm working with LightGBM on a large data set about 3M row and about 8 columns. When i ...
0
votes
0answers
11 views

Can we use both ridge-lasso and PCA in the same model for better results?

My question here is if we are using the PCA, the dimensionality is reduced and no question of feature selection is required using ridge & lasso. So should I use ride-lasso followed by PCA or I ...
1
vote
1answer
29 views

Using ARIMA parameters when transforming time series to Supervised Learning

When forecasting time series one can change the problem from a classical time series (ARIMA type of models) to supervised learning (by adding lag features). When the time series is long and you ...
4
votes
1answer
98 views

Which target variable should I use?

I have a problem where I want an LSTM to predict the resistance of a body. This value can also be calculated if we know the drag coefficient and the speed of that body. In my case, at inference time, ...
0
votes
0answers
14 views

Apply feature importance obtained from entire dataset to individual row

There are various methods for calculating feature importance. These are generally obtained from computing the entire dataset. Can this feature importance be then applied to specific rows? In other ...
0
votes
0answers
27 views

Plotting incremental feature selection curve in WEKA

I have learned that, there are three methods for feature selection in machine learning Filter method Wrapper method Embedded method I want to plot incremental feature selection curve in WEKA ...
0
votes
0answers
11 views

I want to replace XGBRegressor with a simple model to make feature selection

I will make some for loop on to select the best features by my Data frame is big 10M row and about 50 columns so if i replaced xgb with a single Decision tree would it be the same results for the best ...
0
votes
0answers
16 views

ACO Feature Selection for Classifying Images

I have trouble understanding how ant colony optimization works for feature selection in case of image classification. Consider the following graph: where the nodes represent the selected features and ...
0
votes
0answers
24 views

Feature Selection for highly correlated feature

I have used a feature which has a high correlation with the target of 0.8 , but the accuracy of the model decreases in validation set when I add this feature. What might be the reason for decrease in ...
1
vote
1answer
84 views

Why linear regression feature coefficients become super large?

Introduction I've implemented linear regression using sklearn and after all calculations I've got results like this: ...
0
votes
0answers
9 views

How do you determine cut-off values for correlation when choosing features to keep?

As a beginner reading through the literature available on google, it looks like white papers publish WILDLY different scales for what is weak, moderate or strong association. I assume this is because ...
0
votes
0answers
5 views

Using Kendall's Tau for association between dichotomous nominal and ordinal features

I'm working on the titanic data set and I've split my data into 3 groups: ...
0
votes
0answers
12 views

Negative impact of “important” features on model performance

I have a random forest regressor with a set of base features, fit & optimised with sklearn random search algorithm. When I add a set of additional features and retrain (again with random search ...
3
votes
2answers
80 views

Decision Trees and Feature Selection

I'm trying to experiment with the performance of different machine learning algorithms before and after applying feature selection. I tested SVM, Random Forest, KNN, Linear Regression, and, Decision ...
0
votes
0answers
15 views

Which statistical method to use for feature selection between numerical inputs and categorical input?

I have a classification problem where my inputs are all numerical and continuous, my outputs are categorical labels [1,0,-1]. My own domain knowledge and ...
0
votes
1answer
67 views

How to perform feature selection with Categorical Variables and Continuous Target, provided that data is not normally distributed?

Basically I am trying a use Multi Linear Regression Model to predict the salaries of employees. I have a total of 88 dependent feature from which 19 are categorical and the rest are continuous. I have ...
1
vote
1answer
31 views

Creating a composite score from dataset with no target variable

I have a dataset that includes 6 variables about prospective sales opportunities (probability of closing, days until expected close, age of opportunity, etc.). 2 of the columns are categorical and 4 ...
2
votes
0answers
14 views

Adding high p-value and low R square features in linear regression model to improve result

I am working on a linear regression problem. The features for my analysis have been selected using p-values and domain knowledge. After selecting these features, the performance of $R^2$ and the $...
2
votes
1answer
116 views

How to interpret a specific feature importance?

Apologies for a very case specific question. I have a dataset of genes, with which I am using machine learning to predict if a gene causes a disease. One of the features I have is a beta value (which ...
0
votes
1answer
20 views

Relative feature importance w.r.t hyperparameters

Could changing the hyperparameters of a model change relative feature importance?
0
votes
0answers
13 views

Can I perform Verification and validation checks on datasets like AndroPRAguard, Drebin(contain malware and benign mobile apps)?

Verification and validation checks for data: Verification of data: - Visual Checks: It checks data visually. Double Entry Check: It checks duplication of data in the database. Validation of data: ...
0
votes
0answers
14 views

Variable with extra small Pearson coefficient has bigger positive impact on ML model performance than variable with bigger Pearson

I made some machine learning models using Python sci-kit learn library and I found some strange situation for me regarding the real importance of some variables (features) to the ML model. I found ...
1
vote
0answers
17 views

Feature engineering one step at a time or in bunches?

Currently, I'm working on my very first classification project. If you want to know what dataset I'm working with, think "playing stairway to heaven in your local guitar store", and it will probably ...
0
votes
1answer
28 views

Multi-class classification with only one feature

I am studying the efficacy of using a single feature for predicting a set of events (which is a multi-class classification problem). I was wondering if it makes any sense to use only one feature for ...
1
vote
1answer
40 views

using feature selection to improve model performance

I have a highly sparse dataset that I am using to predict a continuous variable via a random forest regression. I have achieved an acceptable level of performance following cross-validation, and I am ...
1
vote
2answers
36 views

Different approaches to label data

I have a dataset of patient records but they don't have labels. I would like to label them and would like to know what are the different approaches available that I can consider to label them. For ...
0
votes
3answers
36 views

Why to exclude features used for label generation during modeling?

I have a dataset like below without labels But with the help of experts opinion, we generate labels based on the below 3 rules (all 3 rules has to be met to label it as 1) So now the dataset looks ...
0
votes
1answer
37 views

How to create a feature vector given final set of feature maps?

I've got a faster-rcnn (resnet-101 backbone) for object detection, and am extracting feature tensors for each detected object, which is a 7x7x2048 tensor (basically 2048 feature maps, each 7x7). For ...
0
votes
1answer
25 views

What are some significance tests to rank features(multiple) before training the data

I have 8 features for a classification problem. The target value tells if there was an anomaly or not. I want to run some significance tests to rank each feature, as being a distinctive feature of ...
1
vote
2answers
149 views

Scikit-learn OneHotEncoder effect on feature selection

If I need to run feature selection on my dataset isn't it problematic to use OneHotEncoder? Couldn't it then decide to remove a one of the encoding columns? How should I deal with this? Thank you.
3
votes
1answer
401 views

Feature selection for data with both continuous and categorical features?

I am working on a classification problem with 4 ordinal classes to predict, labelling/predicting samples as either a number from 1-4. My training dataset has 284 features by ~40,000 samples and I am ...
1
vote
1answer
21 views

In feature selection, I came across a situation where NaN were filled by median of the column values

Why the median value is used for NaN? Why not something else like mean? What is the logic behind using the median value?
1
vote
0answers
26 views

Why not use constant instead of permutation for model agnostic predictor importance?

I want to determine predictor importance. Ideal is to re-train same model on same dataset missing each variable in turn. This is too time consuming. The recommendation I have seen everywhere is to "...
1
vote
0answers
24 views

Does EDA helps only in case of linear regression?

I know what Explanatory data analysis is and how it helps us investigate and understand the data. What I dont understand is how does this help in case of nonlinear relationships? I mean if I'm using ...
2
votes
0answers
22 views

Which stage should the correlation analysis be done?

I was thinking about it, but I couldn't find a logical explanation. Mostly im following below steps after data become ready: Correlation analysis and elimination Apply dummy if categorical variables ...
2
votes
1answer
49 views

Random Forest workflow?

I have a data-set comprised of a fairly large number of columns (over 1000) relative to the number of rows (370) that I am currently running a random forest regression on. I am a little confused with ...
1
vote
1answer
20 views

How to do backward features elimination when considering interactions between them

I have a multi linear regression problem, $Y$ is my target and $X_1, X_2, X_3$ are my features. In my regression, I consider the interaction between $X_1, X_2, X_3$ and I add a bias. So my problem ...
1
vote
0answers
13 views

What toolbox to use to create multi-output random forest(reggression) with custom spltting function at each node?

I am trying to implement "Real Time Head Pose Estimation fromConsumer Depth Cameras" by Fanelli et al. I need to train a random forest(regression) with the following criterion The predicted output is ...
2
votes
1answer
55 views

Correlation based Feature Selection vs Feature Engineering

I'm a bit confused about the superiority of Feature Selection over Feature Engineering or vice versa. Let's say I just want to get the best possible performance on a couple of models like a neural ...
2
votes
1answer
1k views

How to get feature importance from a keras deep learning model?

In case of scikit-learn's models, we can get feature importance using the relevant attributes of the model. I've been working on a RNN, using LSTMs for text embedding. Is there any way to get ...
1
vote
0answers
33 views

Model-independent measures for feature importance given highly correlated features

I am currently working on a research project where the central question is which features drive the prediction of different models. The main issue is, that there is high (multi-)collinearity among ...
0
votes
1answer
29 views

What is the name of this statistical interaction?

What is the name of the following statistical / informational interaction: given A, I know exactly what B is. given B, I know to some extent what A is. I'm not looking for a probability but rather ...
1
vote
0answers
17 views

Do you need to perform variables reduction for tree-based models?

I know for methods and linear regression, GLM, Logistic regression, we typically run through a lot of variable reduction methods, i.e, forward/backward/stepwise, univariate analysis; variable ...

1
2
3 4 5
13