Skip to main content
13 votes

List of feature engineering techniques

Missing Data Imputation: Complete case analysis Mean / Median / Mode imputation Random Sample Imputation Replacement by Arbitrary Value Missing Value Indicator Multivariate imputation ...
Sole G's user avatar
  • 281
11 votes

How to combine categorical and continuous input features for neural network training

There's three main approaches to solving this: Building two models separately and then training an ensemble algorithm that receives the output of the two models as an input Concating all the data ...
Tadej Magajna's user avatar
10 votes

List of feature engineering techniques

There is no definite source on how to do feature engineering. It is often dependent on the problem you are trying to solve. Some say it is more of an art than it is science. But I would go through ...
phiver's user avatar
  • 718
9 votes
Accepted

Combining Latitude/Longitude position into single feature

A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice. Representing latitude and longitude in a single linear scale is not possible due to the fact ...
Julio Cezar Silva's user avatar
8 votes

Are there any tools for feature engineering?

Featuretools is a recently released python library for automated feature engineering. It's based on an algorithm called Deep Feature Synthesis originally developed in 2015 MIT and tested on public ...
Max Kanter's user avatar
8 votes

How to deal with categorical feature of very high cardinality?

This is an old question. I am surprised that I don't see anyone mentioned Mean Encoding (a.k.a Target Encoding). It is very popular in supervised learning problems. Besides, I have seen people use ...
Diansheng's user avatar
  • 181
7 votes

Is this a good practice of feature engineering?

1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
Fansly's user avatar
  • 71
6 votes
Accepted

Is this a good practice of feature engineering?

If you can keep adding new data (based on a main concept such as area i.e. the ZIP code) and the performance of your model improves, then it is of course allowed... assuming you only care about the ...
n1k31t4's user avatar
  • 14.9k
5 votes
Accepted

Convolutional Neural networks

Why don't we convolve our images against the last convolution layer and see how many of these complex feature filters get activated? The answer is that all the layers are fully dependent on the exact ...
Neil Slater's user avatar
5 votes
Accepted

How to transform raw data to fixed-frequency time series?

This sort of effect can be achieved with pandas.DataFrame.resample() combined with Resampler.aggregate() like: Code: ...
Stephen Rauch's user avatar
  • 1,783
4 votes

Too much inputs = overfitting?

Yes, you can mix any different sort of inputs when the scales of the features are similar, which is achieved by normalising the feature vectors. I assume you mean too many features when you say 'too ...
ab123's user avatar
  • 205
4 votes

Is this a good practice of feature engineering?

Usually, the richer the features the better. One thing to keep in mind, however, regressions, in general, do not work well with data that is highly correlated (multicollinearity). When you expand ...
The Lyrist's user avatar
4 votes
Accepted

Is it a good idea to train with a feature which value will be fixed in future predictions?

Given that in your training data this feature has different values and some predictive power, I think not keeping this feature would be a mistake (without looking into overfitting due to having too ...
Jan van der Vegt's user avatar
4 votes

What are features for state-action pairs in RL?

A feature vector is a vector that is containing basis functions. These basis functions are combining states and actions. We can use a feature vector to approximate our action-value function $q(\...
MachineLearner's user avatar
3 votes

How to deal with categorical feature of very high cardinality?

You can use embedding which is mentioned in the comments. e.g. A general blog post, Keras documentation for embedding layer which can be used to learn the embedding. This is widely used by deep ...
eSadr's user avatar
  • 131
3 votes
Accepted

how to evaluate feature quality for decision tree model

The main reasons for seeking an efficient feature selection are the machine learning algorithm get faster training, reduces the complexity of a model, facilitates interpretation and improves the ...
Marcelo's user avatar
  • 46
3 votes
Accepted

What are features for state-action pairs in RL?

In the cartpole example, a state-action feature could be $$\begin{bmatrix} \text{Cart Position}\\ \text{Cart Velocity}\\ \text{Pole Angle}\\ \text{Pole Tip Velocity}\\ \text{Action} \end{bmatrix}$$ ...
Philip Raeisghasem's user avatar
3 votes

Combining Latitude/Longitude position into single feature

The best practice is to not attempt to flatten Earth into a onee dimensional line... Because as you may know, Earth more resembles a sphere than a line. It is much better to treat it as such properly. ...
Has QUIT--Anony-Mousse's user avatar
3 votes

Finding if an outcome is predictable

That would be a part of feature selection. There are many methods to find out if there are relationships between the dependent variable and independent variables. To name a few: plots, measures of ...
Mateusz's user avatar
  • 125
2 votes

Are there any tools for feature engineering?

Scikit-learn has recently released new transformers that tackle many aspects of feature engineering. For example: You can do multiple missing data imputation techniques with the ...
Sole G's user avatar
  • 281
2 votes
Accepted

Time-stamp for linear model

Welcome to Datascience.SE! Like you said, you can extract the day of the week. Also extract the hour of the day, then encode these two variables using sines and cosines with their respective ...
Emre's user avatar
  • 10.5k
2 votes
Accepted

Why adding combinations of features would increase performance of linear SVM?

Multiplication is not a linear operation. Your linear SVM constructs a (hyper-)plane $$ w_0 = w_1 x_1 + w_2 x_2 $$ for some weights $w_0, w_1, w_2.$ By introducing the AND-feature, you add another ...
Elias Strehle's user avatar
2 votes
Accepted

Classification: How to manage data sets where one data row depends on another data row

I assume that you are solving a supervised classification problem, that is, you train your model on a labeled sample. I can think of two approaches to this problem. I. Classify tags, use neighbor ...
David Dale's user avatar
  • 1,551
2 votes

How to combine categorical and continuous input features for neural network training

I had been using the naive structure proposed by you for quite some time by now. In a well framed problem and with enough data, this type of architecture works quite well. However here are a few ...
Soumyajit's user avatar
  • 121
2 votes

how to evaluate feature quality for decision tree model

Another approach to evaluate features is called Permutation Importance. In short, this approach is random sampling values of each feature and each time measuring the negative impact this has on the ...
Franziska W.'s user avatar
2 votes

Creating a metric based on some features

Indeed, there are methodologies that have been tested elsewhere, some with greater and less success. I will propose one of them to build a prediction of job satisfaction, which you can then enter as ...
iair linker's user avatar
2 votes

Importance of features

I'm not sure irrelevant is the right word, but here is an example where it may not be an easy trend. You're trying to predict blood pressure. If you're only given weight, there may be some ...
Sean Payne's user avatar
2 votes

Encode 10k features where each feature is having more than 500 categories

There is no single best technique for anything. You will have to try multiple techniques and see which one gives the best result. Also since your categorical variables are all high cardinal variables, ...
spectre's user avatar
  • 2,115
1 vote
Accepted

I want to create an additional feature(column) based on some manipulation of values from existing features

Welcome to the community! The code below is a starter. You can go on by naming the column and adding it to the original DataFrame: ...
Kasra Manshaei's user avatar
1 vote

Using historical label as a feature in my ML model?

Correct me if I'm wrong, but it sounds like you are using an EMA to generate the price that you will be using to label as either 1,0, or -1. If so, that is not a problem to do so with historical data, ...
stefanLopez's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible