Questions tagged [feature-selection]

Methods and principles of selecting a subset of attributes for use in further modelling

Filter by
Sorted by
Tagged with
2
votes
2answers
855 views

Detecting redundancy with Pearson correlation in continuous features

I have a set of variables that I want to use for a regression or a classification problem. Having computed the correlation matrix of these variables, I discovered that some of them has an inter-...
5
votes
2answers
8k views

Dissmissing features based on correlation with target variable

Is it valid to dismiss features based on their Pearson correlation values with the target variable in a classification problem? say for instance I have a dataset with the following format where the ...
19
votes
3answers
11k views

How to perform feature engineering on unknown features?

I am participating on a kaggle competition. The dataset has around 100 features and all are unknown (in terms of what actually they represent). Basically they are just numbers. People are performing ...
11
votes
4answers
3k views

Which one first: algorithm benchmarking, feature selection, parameter tuning?

When trying to do e.g. a classification, my approach currently is to try out various algorithm first and benchmark them perform feature selection on the best algorithm from 1 above tune the ...
6
votes
1answer
788 views

Interpreting the results of randomized PCA in scikit-learn

I'm using scikit-learn to do a genome-wide association study with a feature vector of about 100K SNPs. My goal is to tell the biologists which SNPs are "interesting". RandomizedPCA really improved ...
1
vote
1answer
140 views

Expand or compact features?

I have a classification task for people with 3 categories. I want to apply machine learning for that. I have 10 sources of data, which have the same fields (say 4: age, job title, a number of ...
5
votes
4answers
329 views

General way to reduce features

Let's say I have a giant dataset (600+ columns) and I have no idea what insights I might get from it or what model I want to run. What are some of the best ways to find the most influential columns/...
2
votes
1answer
101 views

What features most contribute for label y?

On a neural network, how can I find out what features most contribute for some label (the dependent variable y)?
1
vote
1answer
366 views

features' range in logistic regression

I use logistic regression. We know that it is a supervised method and needs calculated feature values both in training and test data. There are six features. Although the functions produce these ...
1
vote
1answer
67 views

Feauture selection for clustering regarding zero-correlated feature

I want to cluster a 5 feature data-set. Firstly to explore the data I did a correlation matrix to see if some features where highly correlated so I could reduce them. Then I saw a feature that have ...
6
votes
2answers
1k views

Improve a regression model and feature selection

I am working on Azure ML Studio and try to create a regression model to predict a numerical value. I will try to describe my features and what I have done until now. My data with about 3 million rows ...
1
vote
0answers
42 views

You are in charge of investing Chipotle's E-Coli source/s, what methods do you use?

With the amount of geographic diversity that is popping up over time with Chipotle and the outbreak of e-coli, if you were investigating the source of all these issues, and where it could occur again ...
3
votes
4answers
1k views

Extremely dominant feature?

I'm new to datascience. I was wondering how one should treat an extremely dominant feature. For example, one of the features is "on"/"off", and when it's "off", none of the other features matter and ...
9
votes
2answers
3k views

What to do when testing data has less features than training data?

Let's say we are predicting the sales of a shop and my training data has two sets of features: One about the store sales with the dates (the field "Store" is not unique) One about the store types (...
3
votes
1answer
2k views

Is there any difference between feature extraction and feature learning?

It appears to me that "feature extraction" and "feature learning" are equivalent concepts, however there are 2 separate wikipedia articles dedicated to them that are notably different. In particular, ...
3
votes
1answer
528 views

How to visualize data of a multidimensional dataset (TIMIT)

I've built a neural network for a speech recognition task using the timit dataset. I've extracted features using the perceptual linear prediction (PLP_ method. My features space has 39 dimensions (13 ...
4
votes
3answers
624 views

Measure of correlation for term frequency

I'm trying to write a framework to compare a set of labels such as (for a sample of 5 yes/no answers to a question) [0, 1, 1, 1, 0] to a series of features to ...
1
vote
1answer
92 views

Use of Correlation score

How do we use a correlation score between two variables for analysing data? I have a set of 20 features and need to predict 21st feature. Now is it necessary that correlation between any two features ...
29
votes
6answers
8k views

Are there any tools for feature engineering?

Specifically what I am looking for are tools with some functionality, which is specific to feature engineering. I would like to be able to easily smooth, visualize, fill gaps, etc. Something similar ...
5
votes
1answer
154 views

Predictive models with class value belonging to a set of observations

I would like to know whether it's possible to build a predictive model where I could define a set of rows with their attributes, and a class belonging to that set of rows, instead of having the ...
3
votes
1answer
236 views

What are the features to distinguish two short term signals (time series)

Imagine that I have a set of short term time series data (signals) with different shapes and parameters. I would like to extract/select a set of features (parameters/functions/indeces) to identify ...
12
votes
1answer
15k views

Feature selection using feature importances in random forests with scikit-learn

I have plotted the feature importances in random forests with scikit-learn. In order to improve the prediction using random forests, how can I use the plot information to remove features? I.e. how to ...
0
votes
2answers
1k views

Identifying top predictors from a mix of categorical and ordinal data

I have a dataset with 261 predictors scraped from a larger set of survey questions. 224 have values which are in a range of scale (some 1-10, some 1-4, some simply binary, all using 0 where no value ...
9
votes
1answer
779 views

Feature selection for Support Vector Machines

My question is three-fold In the context of "Kernelized" support vector machines Is variable/feature selection desirable - especially since we regularize the parameter C to prevent overfitting and ...
4
votes
2answers
3k views

What features from sound waves to use for an AI song composer?

I am planning on making an AI song composer that would take in a bunch of songs of one instrument, extract musical notes (like ABCDEFG) and certain features from the sound wave, preform machine ...
1
vote
0answers
46 views

Trying to come up with a feature to improve emotion classifier based on facial movement using facial landmarks

I managed to create an emotion recognition system that uses dense optical flow on each entire frame. While the accuracy range is within 80-90% with cross-validation, I am aiming to improve the ...
10
votes
7answers
2k views

Data science projects explained step by step?

I am looking for a website or book where several practical examples are given step by step, explaining how they choose the relevant features, the model selection procedure, etc...
1
vote
0answers
85 views

How to include class as a feature

I am currently experimenting with the idea of including the class of a feature vector as a separate feature. My work is about preposition selection in learner language use. I want to train a ...
2
votes
1answer
495 views

How to select features from text data?

I have a data set of questions belonging to ten different categories namely (definitions, factoids, abbreviations, fill in the blanks, verbs, numerals, dates, puzzle, etymology and category relation). ...
5
votes
3answers
712 views

Is automatic feature detection feasible?

I am searching for pointers to algorithms for feature detection. EDIT: all the answers helped me a lot, I cannot decide which one I should accept. THX guys! What I did: For discrete variables (i.e....
4
votes
3answers
964 views

How to find the input variables for a classification problem?

I am working on a classification problem. I have 1000+ features in this dataset. I don't know how to select the right variables/ features that can actually contribute to predicting the output. What ...
1
vote
1answer
150 views

design pattern for extracting features

I am looking for a design pattern that is relevant to a module that extracts features. I want to define a certain number of features over my data points, and then according to the performance and the ...
1
vote
1answer
171 views

Does high error rate in regression imply the data set is unpredictable?

I have a data set of video watching records in a 3G network. In this data set, 2 different kind of features are included: user-side information, e.g., age, gender, data plan and etc; Video watching ...
1
vote
1answer
27 views

Shifting dataPoints up by a constant (Is there an issue with too many 0's for features?)

I am currently collecting second by second data regarding buyer vs seller initiated trades for different financial instruments (securities mostly). If there are more buyer initiated trades in a given ...
2
votes
1answer
51 views

Does a NB wrapper consider feature subset size?

while comparing two different algorithms to feature selection I stumbled upon the follwing question: For a given dataset with a discrete class variable we want to train a naive bayes classifier. We ...
5
votes
2answers
1k views

Time series prediction

I am trying to predict a time serie from another one. My approach is based on a moving windows. I predict the output value of the serie from the following features: the previous value and the 6 past ...
1
vote
2answers
166 views

Time series: variations as a feature

I am trying to predict clients comportement from market rates. The value of the products depends on the actual rate but this is not enough. The comportement of the client also depends on their ...
3
votes
1answer
2k views

Machine Learning for hedging/ portfolio optimization?

With increasingly sophisticated methods that work on large scale datasets, financial applications are obvious. I am aware of machine learning being employed on financial services to detect fraud and ...
8
votes
1answer
1k views

Document classification: tf-idf prior to or after feature filtering?

I have a document classification project where I am getting site content and then assigning one of numerous labels to the website according to content. I found out that tf-idf could be very useful ...
1
vote
0answers
108 views

scikit-learn OMP mem error

I tried to use OMP algorithm available in scikit-learn. My net datasize which includes both target signal and dictionary ~ 1G. However when I ran the code, it exited with mem-error. The machine has ...
47
votes
10answers
41k views

Machine learning - features engineering from date/time data

What are the common/best practices to handle time data for machine learning application? For example, if in data set there is a column with timestamp of event, such as "2014-05-05", how you can ...
13
votes
2answers
4k views

What features are generally used from Parse trees in classification process in NLP?

I am exploring different types of parse tree structures. The two widely known parse tree structures are a) Constituency based parse tree and b) Dependency based parse tree structures. I am able to ...
19
votes
2answers
11k views

Text categorization: combining different kind of features

The problem I am tackling is categorizing short texts into multiple classes. My current approach is to use tf-idf weighted term frequencies and learn a simple linear classifier (logistic regression). ...
4
votes
1answer
799 views

Understanding output stepAIC

I am using the stepAIC function in R to do a bi-directional (forward and backward) stepwise regression. I do not understand what each return value from the function ...
38
votes
5answers
63k views

Does scikit-learn have forward selection/stepwise regression algorithm?

I'm working on the problem with too many features and training my models takes way too long. I implemented forward selection algorithm to choose features. However, I was wondering does scikit-learn ...
16
votes
2answers
5k views

How to choose the features for a neural network?

I know that there is no a clear answer for this question, but let's suppose that I have a huge neural network, with a lot of data and I want to add a new feature in input. The "best" way would be to ...
6
votes
1answer
2k views

How to normalize results of Singular Value Decomposition (SVD) between 0 and 1?

I'm building a recommender system and using SVD as one of the preprocessing techniques. However, I want to normalize all my preprocessed data between 0 and 1 because all of my similarity measures (...
11
votes
4answers
3k views

Feature Extraction Technique - Summarizing a Sequence of Data

I often am building a model (classification or regression) where I have some predictor variables that are sequences and I have been trying to find technique recommendations for summarizing them in the ...
14
votes
4answers
2k views

What are the implications for training a Tree Ensemble with highly biased datasets?

I have a highly biased binary dataset - I have 1000x more examples of the negative class than the positive class. I would like to train a Tree Ensemble (like Extra Random Trees or a Random Forest) on ...
9
votes
1answer
176 views

Learning signal encoding

I have a large number of samples which represent Manchester encoded bit streams as audio signals. The frequency at which they are encoded is the primary frequency component when it is high, and there ...