Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
0
votes
0answers
19 views

Multi-level data, what is the best approach?

I'm working on a dataset and having some problems. I hope you can give me your insight. So my objective is to predict customer churn based on incidents. Each incident is related to a contract which ...
3
votes
2answers
41 views

Decision Trees Should We Discard Low Importance Features?

I just started to work with feature selection. Let's say I have a decision tree model. I get its feature importances by tree.feature_importances_. In my model out ...
2
votes
3answers
90 views

Can ridge regression be used for feature selection?

I'm trying to figure out whether using Ridge Regression for regularization can be used to cause a more sparse hypothesis however to me it seems like ridge will never actually bring any coefficients to ...
1
vote
2answers
37 views

Feature selection filter methods

I am confused about when to use which filter methods for feature selection. I tried to learn them through online resources and found methods like chi-square, variance threshold, F-test, Mutual ...
2
votes
1answer
140 views

SelectKBest and Correlation returns me excatly same feature selection. How?

Im working on selecting most effective features from a dataset with over that 2000 features. Im using different algorithms for that (selectKBest with chi-square, Extra Trees, Correlation etc.) But ...
2
votes
2answers
99 views

Using random forest for selecting variables returns the entire dataframe

I am in the process of dimensionality reduction. I am using Random Forest to find the columns with the highest level of correlation with the target SalePrice column. The problem is that the output ...
0
votes
1answer
45 views

How to input a 3d model into ML algorithm?

I have a machine learning model that uses csv with measured data about buildings: width, length, height etc. I use it to predict some features and it works properly. I would like to drop csv with ...
1
vote
1answer
234 views

Light GBM Regressor, L1 & L2 Regularization and Feature Importances

I want to know how L1 & L2 regularization works in Light GBM and how to interpret the feature importances. Scenario is: I used LGBM Regressor with RandomizedSearchCV (cv=3, iterations=50) on a ...
0
votes
0answers
24 views

Feature selection for circular data in time-series

I'm predicting ozone concentration based on meteorological and seasonal variables. In the feature engineering stage I converted the MONTH, DAY_OF_WEEK, DAY_OF_YEAR to its sin and cosine components ...
2
votes
1answer
197 views

How does SHAP values help us to determine importance of a feature for a model trained by gradient boost?

I've read http://papers.nips.cc/paper/7062-a-unified-approach-to-interpreting-model-predictions.pdf and https://medium.com/@gabrieltseng/interpreting-complex-models-with-shap-values-1c187db6ec83 which ...
0
votes
1answer
22 views

Multivariate Multilag Regression with one shot prediction using LSTM

I am working on a multivariate regression task using a LSTM and I am interested in one shot prediction of my target variable (which is the price of a commodity). For example, the first parameter I ...
0
votes
1answer
29 views

Feature selection before or after applying filter in Time-series forecasting

I'm predicting ozone concentration based on meteorological variables and ozone value of the previous day. I applied savitzky golay filter to get rid of noise in the time-series dataset. My question ...
2
votes
1answer
43 views

Can anyone explain me the fisher score working

I have been working on feature selection and I wanted to know what does fisher score tell us about the data which helps us in feature selection.
0
votes
0answers
14 views

What are the common feature extraction technic to compare 2 sequence of timestamps?

I am building some predictive models for an online shopping site. I have timestamp log of customers' arrival time, time spent on a product's webpage, purchased or not, and a few others. I have tried ...
0
votes
1answer
17 views

How to include both origin and destination in your features?

I'm trying to predict the price of transportation for trucking freight. Two important features that I think would be of great impact are Origin and Destination. What's the best way to include that in ...
1
vote
1answer
27 views

For a regression model, can you transform all your features to linear to make a better prediction?

I was thinking. Would it be a good approach to check your features one by one (assuming you have a manageable amount of them) and see the relationship they have with your target variable, if they ...
3
votes
1answer
32 views

CrossValidation using glmnet and very high values of Lambda?

I am trying to run crossvalidation (folds=10) using glmnet library on my dataset. My outcome of interest is BMI and predictors include a set of clinical variables. My final goal is to use elastic-net ...
2
votes
2answers
80 views

unimportant features impact on model's performance

Using XGBoost and RandomForests, do unimportant features (according to the feature_importances_ attribute) hurt the model's performance? Do I need to carefully ...
1
vote
1answer
37 views

Using of 100s of Binary features in regression model

I have 100s of columns with binary values [0, 1] plus some extra columns without binary values. I am trying to do regression model but the model performance is very low. For non-binary features, I ...
0
votes
0answers
16 views

When we use VarianceThreshold from sklearn and do the transform so does our data also get scaled

I was working with Variance Threshold and when I used the transform function ,I found that the output is in floating points where as in original data set it was in integer format so why does it happen....
0
votes
0answers
24 views

records with perfect correlation to the answer. Drop or Keep?

I have about 1000 records (5 numeric, 5 categorical vars) and about 25 of them have something in 5-level categorical variable that just gives away answer. It's just too obvious and I'm not worried ...
0
votes
1answer
20 views

Identifying importance of each feature in deep model

I have a deep model and I want to figure out which feature has the maximum influence on predicted result. For this I train the model with all the features I think are important, during prediction I ...
0
votes
1answer
18 views

Using strong predictor in Model training?

I am trying to build a Disease predictor based on symptoms. I am using data scraped from Symcat website. After sampling the data we have symptoms to disease mapped for training purpose.Data looks like ...
1
vote
1answer
30 views

Unable to understand which features to choose

I am a newbie here, but I am trying to work with a dataset which gives the attempt at the goal by a footballer,which will predict one of 2 possible outputs - whether or not they could score the goal ...
1
vote
1answer
49 views

How to use Random Forest to reduce dimensions

I am working on the Boston competition on Kaggle and at the moment I am trying to use Random Forest to find the columns with the highest correlation with the target variable ...
0
votes
1answer
34 views

Getting a ValueError from train_test_split

I'm working on this dataset. I'm trying to select features using Random Forest. This is the relevant code: ...
1
vote
1answer
35 views

Feature selection method explanation

In the context of feature relevance, I am trying to understand the meaning of the correlation method for feature selection. Can somebody please explain if the following results of the correlation ...
0
votes
1answer
41 views

Which feature to use in feature selection?

Objective: Multiclass classification with supervised learning, small dataset (25h) Context: My dataset is composed of mobile network data collected with a smartphone. The labels correspond to the ...
0
votes
0answers
3 views

Not sure how to use GLCM features to clasify large cuantities of textures

Im trying to create a script that given a texture returns similar textures. I have read some articles and I have found that, for textures, good ways to obtain features are to use LBP or GLCM. I have ...
1
vote
2answers
77 views

Regression Algorithms in Production

I am interested in predicting if a doctor would prescribe a specific drug and have chosen Logistic Regression as a starting point. I have a few questions: Is feature selection the first step to take ...
0
votes
1answer
75 views

How to handle large number of categories in a dataset?

I have one dataset of "Books" which contains 8 columns initially and out of which 3 of them contains text values which can be categorized. The 3 columns contains "Language-code", "Author Name" and "...
0
votes
0answers
17 views

Feature Engineering and Time calculation for grid search cv

I am new to data science and don't have model building experience. I have a dataframe with 5000 rows and 10 columns. The target column takes 1/0 as values. One feature column is ZIP Code. I converted ...
4
votes
1answer
39 views

In Conditional Random Fields, is mandatory to use features related to following and preceeding tokens?

I am training a CRF classifier to classify document rows as a heading (1st level), heading (2nd level) or simple text. I am using Conditional Random Fields for their ability to account sequential ...
1
vote
1answer
20 views

ML Approach for Getting List of Observations with Similar Features (Discrete+Continuous)

I have a dataset with 19k observations. Each has approximately 448 features: - Text description turned into vectors of size 300 - 16 categorical variables represented numerically - The remainder ...
1
vote
0answers
48 views

Feature Importance Scores from Gradient Boosting vs Random Forest

In sklearn, the feature_importances_ attribute exists for both RandomForestClassifier and GradientBoostingClassifier. Would like to know what are the fundamental differences in how this attribute is ...
0
votes
1answer
20 views

Train and predict on a varying number of inputs - time based events

I have the problem where I am trying to build a model which takes in n events for a single user as input for prediction, the problem is that the number of events is ...
0
votes
0answers
18 views

How to handle “ordered” features?

I have a dataset with weekly sales figures, and trying to building a classification model (predict stock-out). I want to use some feature(s) to tell the machine that: Week 1 comes after week 0 ...
0
votes
0answers
7 views

Is it worth graphing a correlation plot between your features and your target?

Is it worth graphing the correlation between the most important features and the target variable after you have done either PCA or L1 Regularization to identify the most important features? I guess ...
0
votes
2answers
61 views

How do you apply hypothesis testing to your features?

How do you apply hypothesis testing to your features in a ML model? Let say for example that I am doing a regression task and I want to cut some features (once I have trained my model) to increase ...
0
votes
3answers
53 views

How to interpret feature importance (XGBoost) in this case?

I found two dominant features from plot_importance. My dependent variable Y is customer retention (whether or not the customer will retain, 1=yes, 0=no). My ...
1
vote
1answer
57 views

Feature selection or Dimension reduction in unsupervised learning

I'm trying to do Embedded clustering using kmeans. This is customer data, so it involves a lot of sentences, so I'm using the universal sentence encoder before clustering. But I should be doing a ...
3
votes
4answers
315 views

What does embedding mean in machine learning?

I just met a terminology called "embedding" in a paper regarding deep learning. The context is "multi-modal embedding" My guess: embedding of something is extract some feature of sth,to form a vector....
0
votes
0answers
19 views

Do I discard all my dependent variables as proved by chi-squared test of independence?

I have 134 categorical columns in my data. 7 of which are categorical variables [ one variable is highly unbalanced and has 34 classes while all other variables just has 3-5 classes in each variable ...
2
votes
1answer
35 views

How valuable is a categorical feature that has a predominant category over all other ones?

Is a categorical feature that has almost equally distributed in it's category more important or the one which one of it's category is predominant over all other ones? In data prepossessing step for "...
0
votes
2answers
61 views

Correlation feature selection followed by regression

I have quarterly results data for a company with around 100 variables. Total 60 quarters results are available (total records 60). sample data: (only few columns & 10 rows) I would like know ...
0
votes
1answer
35 views

How to predict specific user from session logs?

Let's say I have a dataset with 800 rows (40 entries for each of 20 users). The entries are user session logs (columns are - browser, os, time, date etc for a specific session). Now each user has ...
0
votes
2answers
47 views

How can i test the performance of a model when the test data contains seen and unseen data

To test the performance of my model based on some selected features, i try to use unseen and seen data. However, when choosing the accuracy based on all data, the model is almost overfitting since ...
0
votes
1answer
23 views

How to interpret a random variable in the variable importance?

I have a problem, for simplicity let's say it is a binary classification problem. I am trying to solve this problem using XGBoost. A standard output plot for any ML algorithm, is the feature ...
0
votes
1answer
151 views

No correlation found between dataset features

I'm trying to build a classification model that predicts the price of New York taxi trips (year 2018). Datasource page Since the original file is very large (112 234 626 rows), I constructed the ...
2
votes
1answer
52 views

Multiclass classification with high number of classes, high number of features and small sample size

I am working on a biology related dataset with over 300K features, and I only have about 5K samples. I want my model to classify many classes. For this problem in particular the class is age. Each age ...