Questions tagged [feature-selection]
Methods and principles of selecting a subset of attributes for use in further modelling
1
vote
1answer
10 views
Boruta Python No feature Selected
I run Boruta with RandomForestClassifier the previous day on my data (nb features = 36) and got 17/36 confirmed. Now I run it again and there is 0/36 and stop at the 9th iteration. Any idea why this ...
5
votes
3answers
58 views
How to handle features which are not always available?
I have a feature in my feature vector that is not always available respectively sometimes (for some samples) it makes no sense to use it. I feed a sklearn MLPClassifier with this feature vector. Does ...
-1
votes
1answer
52 views
ML: How to think feature selection?
What is the basic philosophy behind feature selection and modelling? How do you actually start? Could you please share your real (practical) inputs?
Bit of background:
I am actually trying to analyse ...
0
votes
0answers
17 views
Feature selection using a filter for multiclass problem: What if many features are strongly predictive of few classes?
I'm doing text classification with a bit more than 100 classes. First, I would like to do feature selection by using a filter approach (mutual information or chi2).
I planned on using ...
6
votes
1answer
72 views
Will unnecessary features harm the tree based model?
Is it necessary to drop noisy features (eg column of random numbers) from tree features? I think it's not. sometimes it may benefit but will never cause any harm to the model. Because at each split ...
0
votes
0answers
7 views
Provide optional confidence level as an input to the neural network
I have a name, gender labeled dataset and I know the frequency of particular name can occurred in the dataset. I want to develop a neural network which predict gender when given the name as an input. ...
1
vote
0answers
17 views
What is serialization? [closed]
A fictional Broadway show has 3 shows every Saturday. Tickets are valid for a particular show and enumerated seat. The process of encoding the showtime and seat number is a unique.
Is the process ...
1
vote
0answers
11 views
Feature selection with information gain (KL divergence) and mutual information yields different results
I'm comparing different techniques for feature selection / feature ranking. Two of the techniques under scrutiny are the mutual information (MI) and the information gain (IG) as used in decision trees,...
0
votes
1answer
26 views
How to select variables based on the mean correlation in a correlation matrix?
I have a set of independent variables and I am calculating the correlation matrix between them using the Pearson Correlation Coefficient in Python. A part of the matrix looks like this:
From this ...
1
vote
3answers
57 views
In linear regression, is there anything I can do if the coefficient for one of the features is unrealistic/inappropriate?
I'm building a simple linear regression model that predicts Home Price using Square Footage, Number of Bed(s), and Number of Bathroom(s).
After creating the model, I noticed that the coefficients for ...
1
vote
0answers
39 views
How to determine feature importance in a neural network?
I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the ...
0
votes
0answers
19 views
Feature selection through Random Forest and Principal Component Analysis
I am working on a binary classification problem and I have 870 numeric independent features to start with. I tried PCA on input features and picked top 200 variables corresponding to first 10 ...
0
votes
0answers
7 views
Knowing Feature Importance from Sparse Matrix
I was working with a dataset which had a textual column as well as numerical columns, so I used tfidf for textual column and created a sparse matrix, similarly for the numerical features I created a ...
2
votes
2answers
67 views
How much data to use for feature selection?
Working on my master's thesis, this is a problem I'm unable to find good resources about.
I'm working with data of 18 participants, who are either active or passive. Each participant is then ...
2
votes
1answer
78 views
When should I use StandardScaler and when MinMaxScaler?
I have a feature vector with One-Hot-Encoded features and with continous features.
How can I decide now, which data I shall scale with StandardScaler and which data scale with MinMaxScaler? I think I ...
0
votes
0answers
5 views
¿Is Weight of Evidencie a good feature selector?
I was reading in Weight of evidence (WoE) and while it seems like an interesting way of assessing feature importance. It seems to me than binning everything may affect the performance by assuming ...
1
vote
1answer
38 views
Why does Feature Importance change with each iteration of a Decision Tree Classifier?
After applying PCA to reduce the number of features, I am using a DecisionTreeClassifier for a ML problem
Additionally I want to compute the feature_importances_. However, with each iteration of the ...
0
votes
0answers
27 views
iris dataset feature selection using cuckoo search optimisation
I am trying to use a python library SwarmPackagePy to select significant features from a dataset for example iris dataset But i am stuck in dimension and iteration setting.
In addition in how to use ...
0
votes
0answers
15 views
ML Approach for Matching Two Binary Feature Arrays
I have a research project that involves this basic problem (paraphrased for discretion and because I love Legos (: )
100 individuals each have 50 Lego instruction manuals, from a total set of 10,000 ...
1
vote
0answers
19 views
Remove attributes with missing values exceeding a given threshold in WEKA
I imported csv file into WEKA, i have features that have missing value that has missing value percentage of 70% or above, i want to remove these features by weka or also sort that features by missing ...
1
vote
0answers
37 views
PCA for unsupervised feature selection [closed]
If I understood correctly, "using results of PCA to select features" (as recommended in this answer) implies visually analysing bi-plots of first two principal components - i.e. the angle between a ...
0
votes
0answers
52 views
Feature importance and probability score out of Decision Trees
We all know that Decision trees are super interpretable but one thing that I am not able to understand is the mathematics behind it. So, I have three questions here :
How do Decision tree and Radom ...
3
votes
3answers
79 views
how to evaluate feature quality for decision tree model
Most of the tutorials assume that the features are known before generating the model and give no way to select 'good' feature and to discard 'bad' ones.
The naive method is to test the model with new ...
0
votes
0answers
11 views
Forward Feature Selection in classification generating same training error
Starting Notes - I am very beginner in data science so it may be possible that i will be doing the very basic things in an incorrect way.
Preview - I am trying to predict the Survived label for the ...
0
votes
0answers
10 views
Reducing Bias when trying to find Feature Importance using a Random Forest
I'm currently looking to show which of three variables is more important in classifying something as True or False. Everyone agrees that all three variables are important, but not all agreeing on what ...
1
vote
1answer
31 views
Who wrote the formula for gini importance/sklearn's feature importance score?
I've been looking for a paper where the Gini importance was first proposed, but I am not sure if this is actually how it came to be.
Here's the formula I am familiar with and am looking to find in a ...
2
votes
0answers
19 views
Structured Support Vector Machine (Joint Feature Map)
I'm studying Structured Support Vector Machine. (https://en.wikipedia.org/wiki/Structured_support_vector_machine)
The theory's clear, but I need a tangible example to make everything more concrete.
...
0
votes
1answer
32 views
How do I right feature selection for DBSCAN?
I want to use DBSCAN to recognize any clusters within all text elements from the DOM tree of any webpage. For example all menu items shall be clustered separatey to all main content or footer elements....
0
votes
0answers
16 views
Low AUC in a Linear Regression Model
I am using AWS Machine Learning service to create my own Linear Regression model using my own dataset, however when the model is created the Evaluation Summary shows a very low AUC of 0.519
The ...
0
votes
0answers
9 views
Feature selection: variance vs. Index of Dispersion
I want to remove some features of my dataset before the training process. I'd like to use the VarianceThreshold method from Sklearn. However, this requires the data to be in the same range 0-1. In ...
0
votes
1answer
17 views
Classification procedure using Minimum Redundancy Maximum Relevance (MRMR)
After selecting features with MRMR (by quantizing original feature space of training data), should we classify the test data using quantized values or original values?
1
vote
0answers
16 views
Show importance of variables from a data set without a response variable? Use PCA? [closed]
I am trying to find a way to statistically show that some variables in my data set are more important than others to determine its classification. I have an example data set with three variables from ...
0
votes
1answer
24 views
Queries regarding feature importance for categorical features
Queries regarding feature importance for categorical features:
Context:
I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, ...
0
votes
2answers
30 views
Hesitate to drop a feature
I used to tried making a fake correlation in my mocking dataset and found that if the score is more than 0.5 I can reduce feature to avoid singularity
In the given ...
1
vote
0answers
29 views
Is there a good online course for working with sklearn MLPClassifier?
I'am implementing an App for which I need a neural network. Because: I want to classify each DOM-text-element of an webpage which conains any curriculum vitae of a person. The neural network should be ...
2
votes
3answers
42 views
Does adding new features that contain information derivable from current features help performance?
So say you have some data that consists of some values:
1.3, 0.9, 1.1
You introduce a new feature which is the average of these values: 3.3
In this example lets say that you know the average of ...
1
vote
1answer
19 views
Feature selection/reduction techinique for combination of features in image processing
I have a combination of features extracted from 3 descriptors, namely GLCM based feaures(correlation, homogeneity,energy and contrast ), Local binary patterns (256) and discrete wavelet transform ...
0
votes
2answers
72 views
Features selection in KNN
I have a naive question about using the K Nearest Neighbor algorithm: is feature selection more important in KNN than in other algorithms?
If a particular feature is not predictive in a neural ...
0
votes
1answer
105 views
How to handle noisy data?
I have X values such as
[0.2, 0.1, 0.3, 0.5, 20, -0.2, -0.1, -0.6, -0.8, -30]
Now I know that if the abs value is below a certain threshold, let's say 2, the y-...
1
vote
1answer
107 views
LightGBM - Why Exclusive Feature Bundling (EFB)?
I'm currently studying GBDT and started reading LightGBM's research paper: https://papers.nips.cc/paper/6907-lightgbm-a-highly-efficient-gradient-boosting-decision-tree.pdf
In section 4. they explain ...
2
votes
2answers
48 views
How can I improve a machine learning model?
I am a Machine learning newbie and i am trying my hands with a dataset which has 9 features and my aim is to figure out the optimal multi class classification model which fits my dataset.
I applied ...
0
votes
1answer
70 views
Concept of Cross-Entropy [closed]
Can anyone say the cross entropy Conceptually in machine learning for selecting best features, in simple way?
I confused about that.
You do not really understand something unless you can explain it ...
0
votes
0answers
13 views
Same feature with different representation
lets suppose that I have two different datasets (A and B), and I want to train a single machine learning model using common features between them.
One of these features called "score". Let's say that ...
1
vote
1answer
23 views
Feature Selection Phase
I am trying to predict the overall age of an opportunity (creation date - closing date) this is my response variable
lets say an opportunity passes through 3 stages
to close
For example: Opp x ...
2
votes
2answers
97 views
Concept of Mutual Information
I want to get mutual information in iris dataset to select best feature but i confused about mutual information.
What is concept of mutual information for selecting feature?
Can anyone explain it in ...
2
votes
0answers
128 views
Combine multiple features for text classification
Recently I started reading more about NLP and following tutorials in Python in order to learn more about the subject. I'm trying to make my own classification algorithm (the text sends a positive/...
3
votes
1answer
50 views
Why would a fake feature with random numbers get selected in feature importance?
I'm using a sklearn.ensemble.RandomForestClassifier(n_estimators=100) to work on this challenge:
https://kaggle.com/c/two-sigma-financial-news
I've plotted my ...
0
votes
2answers
31 views
Rule of thumb for good number of features when dealing with grouped data
I have a classification problem on clinical data where I have multiple samples for each patient. So the samples related to the same patient are somehow dependent from each other.
I know that is not ...
1
vote
0answers
10 views
How do I store/model data needed for my recommendation module?
I'm reading data from a store's product catalog, a 100mb xml file which contains product-wise attributes like prices, categories, etc.
Given a product_id, my job ...
1
vote
1answer
23 views
Effect of adding extra unrelated features to linear perceptron
Suppose that we are training a linear regressor (perceptron). Adding extra features that are not related to the target (e.g. randomly generated values) before training will typically ____ our training ...
