Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
3
votes
1answer
258 views

Explanation of how DeepExplainer works to obtain SHAP values in simple terms

I have been using DeepExplainer (DE) to obtain the approximate SHAP values for my MLP model. I am following https://github.com/slundberg/shap and DE's performance is very high in terms of computation ...
1
vote
2answers
190 views

Finding Feature Importance in CNN's?

Let's say I have images of cars. For each image in the dataset, I have let's say 3 pictures of the same car but in different angles. 1) The first image is the picture of the car from the front. 2) ...
0
votes
0answers
25 views

What is the dimension reduction method to large numbers of independent features while only two of them are important?why?

What is the dimension reduction method to model a data with large numbers of independent features (for instance 5k features), while only two of them are important (are effective in cost function)? I ...
0
votes
0answers
19 views

Multi-level data, what is the best approach?

I'm working on a dataset and having some problems. I hope you can give me your insight. So my objective is to predict customer churn based on incidents. Each incident is related to a contract which ...
3
votes
2answers
44 views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
2
votes
3answers
98 views

Can ridge regression be used for feature selection?

I'm trying to figure out whether using Ridge Regression for regularization can be used to cause a more sparse hypothesis however to me it seems like ridge will never actually bring any coefficients to ...
1
vote
2answers
39 views

Feature selection filter methods

I am confused about when to use which filter methods for feature selection. I tried to learn them through online resources and found methods like chi-square, variance threshold, F-test, Mutual ...
2
votes
1answer
161 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
2
votes
2answers
101 views

Using random forest for selecting variables returns the entire dataframe

I am in the process of dimensionality reduction. I am using Random Forest to find the columns with the highest level of correlation with the target SalePrice column. The problem is that the output ...
0
votes
1answer
65 views

How to input a 3d model into ML algorithm?

I have a machine learning model that uses csv with measured data about buildings: width, length, height etc. I use it to predict some features and it works properly. I would like to drop csv with ...
1
vote
1answer
358 views

Light GBM Regressor, L1 & L2 Regularization and Feature Importances

I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances. Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
0
votes
0answers
28 views

Feature selection for circular data in time-series

I'm predicting ozone concentration based on meteorological and seasonal variables. In the feature engineering stage I converted the MONTH, DAY_OF_WEEK, DAY_OF_YEAR to its sin and cosine components ...
2
votes
1answer
331 views

How does SHAP values help us to determine importance of a feature for a model trained by gradient boost?

I've read http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf and https://medium.com/@gabrieltseng/interpreting-complex-models-with-shap-values-1c187db6ec83 which ...
0
votes
1answer
27 views

Multivariate Multilag Regression with one shot prediction using LSTM

I am working on a multivariate regression task using a LSTM and I am interested in one shot prediction of my target variable (which is the price of a commodity). For example, the first parameter I ...
0
votes
1answer
36 views

Feature selection before or after applying filter in Time-series forecasting

I'm predicting ozone concentration based on meteorological variables and ozone value of the previous day. I applied savitzky golay filter to get rid of noise in the time-series dataset. My question ...
2
votes
1answer
57 views

Can anyone explain me the fisher score working

I have been working on feature selection and I wanted to know what does fisher score tell us about the data which helps us in feature selection.
0
votes
0answers
15 views

What are the common feature extraction technic to compare 2 sequence of timestamps?

I am building some predictive models for an online shopping site. I have timestamp log of customers' arrival time, time spent on a product's webpage, purchased or not, and a few others. I have tried ...
1
vote
1answer
22 views

How to include both origin and destination in your features?

I'm trying to predict the price of transportation for trucking freight. Two important features that I think would be of great impact are Origin and Destination. What's the best way to include that in ...
1
vote
1answer
34 views

For a regression model, can you transform all your features to linear to make a better prediction?

I was thinking. Would it be a good approach to check your features one by one (assuming you have a manageable amount of them) and see the relationship they have with your target variable, if they ...
3
votes
1answer
34 views

CrossValidation using glmnet and very high values of Lambda?

I am trying to run crossvalidation (folds=10) using glmnet library on my dataset. My outcome of interest is BMI and predictors include a set of clinical variables. My final goal is to use elastic-net ...
2
votes
2answers
97 views

unimportant features impact on model's performance

Using XGBoost and RandomForests, do unimportant features (according to the feature_importances_ attribute) hurt the model's performance? Do I need to carefully ...
1
vote
1answer
42 views

Using of 100s of Binary features in regression model

I have 100s of columns with binary values [0, 1] plus some extra columns without binary values. I am trying to do regression model but the model performance is very low. For non-binary features, I ...
0
votes
0answers
17 views

When we use VarianceThreshold from sklearn and do the transform so does our data also get scaled

I was working with Variance Threshold and when I used the transform function ,I found that the output is in floating points where as in original data set it was in integer format so why does it happen....
0
votes
0answers
24 views

records with perfect correlation to the answer. Drop or Keep?

I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
0
votes
1answer
20 views

Identifying importance of each feature in deep model

I have a deep model and I want to figure out which feature has the maximum influence on predicted result. For this I train the model with all the features I think are important, during prediction I ...
0
votes
1answer
21 views

Using strong predictor in Model training?

I am trying to build a Disease predictor based on symptoms. I am using data scraped from Symcat website. After sampling the data we have symptoms to disease mapped for training purpose.Data looks like ...
1
vote
1answer
36 views

Unable to understand which features to choose

I am a newbie here, but I am trying to work with a dataset which gives the attempt at the goal by a footballer,which will predict one of 2 possible outputs - whether or not they could score the goal ...
1
vote
1answer
57 views

How to use Random Forest to reduce dimensions

I am working on the Boston competition on Kaggle and at the moment I am trying to use Random Forest to find the columns with the highest correlation with the target variable ...
0
votes
1answer
61 views

Getting a ValueError from train_test_split

I'm working on this dataset. I'm trying to select features using Random Forest. This is the relevant code: ...
1
vote
1answer
36 views

Feature selection method explanation

In the context of feature relevance, I am trying to understand the meaning of the correlation method for feature selection. Can somebody please explain if the following results of the correlation ...
0
votes
1answer
48 views

Which feature to use in feature selection?

Objective: Multiclass classification with supervised learning, small dataset (25h) Context: My dataset is composed of mobile network data collected with a smartphone. The labels correspond to the ...
0
votes
0answers
4 views

Not sure how to use GLCM features to clasify large cuantities of textures

Im trying to create a script that given a texture returns similar textures. I have read some articles and I have found that, for textures, good ways to obtain features are to use LBP or GLCM. I have ...
1
vote
2answers
78 views

Regression Algorithms in Production

I am interested in predicting if a doctor would prescribe a specific drug and have chosen Logistic Regression as a starting point. I have a few questions: Is feature selection the first step to take ...
0
votes
1answer
103 views

How to handle large number of categories in a dataset?

I have one dataset of "Books" which contains 8 columns initially and out of which 3 of them contains text values which can be categorized. The 3 columns contains "Language-code", "Author Name" and "...
0
votes
0answers
19 views

Feature Engineering and Time calculation for grid search cv

I am new to data science and don't have model building experience. I have a dataframe with 5000 rows and 10 columns. The target column takes 1/0 as values. One feature column is ZIP Code. I converted ...
4
votes
1answer
42 views

In Conditional Random Fields, is mandatory to use features related to following and preceeding tokens?

I am training a CRF classifier to classify document rows as a heading (1st level), heading (2nd level) or simple text. I am using Conditional Random Fields for their ability to account sequential ...
1
vote
1answer
22 views

ML Approach for Getting List of Observations with Similar Features (Discrete+Continuous)

I have a dataset with 19k observations. Each has approximately 448 features: - Text description turned into vectors of size 300 - 16 categorical variables represented numerically - The remainder ...
1
vote
0answers
49 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
0
votes
1answer
22 views

Train and predict on a varying number of inputs - time based events

I have the problem where I am trying to build a model which takes in n events for a single user as input for prediction, the problem is that the number of events is ...
0
votes
0answers
18 views

How to handle “ordered” features?

I have a dataset with weekly sales figures, and trying to building a classification model (predict stock-out). I want to use some feature(s) to tell the machine that: Week 1 comes after week 0 ...
0
votes
0answers
7 views

Is it worth graphing a correlation plot between your features and your target?

Is it worth graphing the correlation between the most important features and the target variable after you have done either PCA or L1 Regularization to identify the most important features? I guess ...
0
votes
2answers
80 views

How do you apply hypothesis testing to your features?

How do you apply hypothesis testing to your features in a ML model? Let say for example that I am doing a regression task and I want to cut some features (once I have trained my model) to increase ...
0
votes
3answers
60 views

How to interpret feature importance (XGBoost) in this case?

I found two dominant features from plot_importance. My dependent variable Y is customer retention (whether or not the customer will retain, 1=yes, 0=no). My ...
1
vote
1answer
72 views

Feature selection or Dimension reduction in unsupervised learning

I'm trying to do Embedded clustering using kmeans. This is customer data, so it involves a lot of sentences, so I'm using the universal sentence encoder before clustering. But I should be doing a ...
3
votes
4answers
570 views

What does embedding mean in machine learning?

I just met a terminology called "embedding" in a paper regarding deep learning. The context is "multi-modal embedding" My guess: embedding of something is extract some feature of sth,to form a vector....
0
votes
0answers
22 views

Do I discard all my dependent variables as proved by chi-squared test of independence?

I have 134 categorical columns in my data. 7 of which are categorical variables [ one variable is highly unbalanced and has 34 classes while all other variables just has 3-5 classes in each variable ...
2
votes
1answer
35 views

How valuable is a categorical feature that has a predominant category over all other ones?

Is a categorical feature that has almost equally distributed in it's category more important or the one which one of it's category is predominant over all other ones? In data prepossessing step for "...
0
votes
2answers
63 views

Correlation feature selection followed by regression

I have quarterly results data for a company with around 100 variables. Total 60 quarters results are available (total records 60). sample data: (only few columns & 10 rows) I would like know ...
0
votes
1answer
36 views

How to predict specific user from session logs?

Let's say I have a dataset with 800 rows (40 entries for each of 20 users). The entries are user session logs (columns are - browser, os, time, date etc for a specific session). Now each user has ...
0
votes
2answers
51 views

How can i test the performance of a model when the test data contains seen and unseen data

To test the performance of my model based on some selected features, i try to use unseen and seen data. However, when choosing the accuracy based on all data, the model is almost overfitting since ...