13
votes
List of feature engineering techniques
Missing Data Imputation:
Complete case analysis
Mean / Median / Mode imputation
Random Sample Imputation
Replacement by Arbitrary Value
Missing Value Indicator
Multivariate imputation
...
- 290
11
votes
How to combine categorical and continuous input features for neural network training
There's three main approaches to solving this:
Building two models separately and then training an ensemble algorithm that receives the output of the two models as an input
Concating all the data ...
- 211
9
votes
Accepted
Combining Latitude/Longitude position into single feature
A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice.
Representing latitude and longitude in a single linear scale is not possible due to the fact ...
8
votes
How to deal with categorical feature of very high cardinality?
This is an old question. I am surprised that I don't see anyone mentioned Mean Encoding (a.k.a Target Encoding). It is very popular in supervised learning problems. Besides, I have seen people use ...
- 181
8
votes
Are there any tools for feature engineering?
Featuretools is a recently released python library for automated feature engineering. It's based on an algorithm called Deep Feature Synthesis originally developed in 2015 MIT and tested on public ...
- 181
7
votes
Is this a good practice of feature engineering?
1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
- 71
6
votes
Accepted
Is this a good practice of feature engineering?
If you can keep adding new data (based on a main concept such as area i.e. the ZIP code) and the performance of your model improves, then it is of course allowed... assuming you only care about the ...
- 15.1k
5
votes
Accepted
Convolutional Neural networks
Why don't we convolve our images against the last convolution layer and see how many of these complex feature filters get activated?
The answer is that all the layers are fully dependent on the exact ...
- 29.2k
5
votes
Accepted
How to transform raw data to fixed-frequency time series?
This sort of effect can be achieved with
pandas.DataFrame.resample() combined with Resampler.aggregate() like:
Code:
...
- 1,811
4
votes
Too much inputs = overfitting?
Yes, you can mix any different sort of inputs when the scales of the features are similar, which is achieved by normalising the feature vectors.
I assume you mean too many features when you say 'too ...
- 189
4
votes
Is this a good practice of feature engineering?
Usually, the richer the features the better.
One thing to keep in mind, however, regressions, in general, do not work well with data that is highly correlated (multicollinearity). When you expand ...
- 484
4
votes
What are features for state-action pairs in RL?
A feature vector is a vector that is containing basis functions. These basis functions are combining states and actions. We can use a feature vector to approximate our action-value function $q(\...
- 1,958
3
votes
Accepted
how to evaluate feature quality for decision tree model
The main reasons for seeking an efficient feature selection are the machine learning algorithm get faster training, reduces the complexity of a model, facilitates interpretation and improves the ...
- 46
3
votes
Accepted
What are features for state-action pairs in RL?
In the cartpole example, a state-action feature could be
$$\begin{bmatrix}
\text{Cart Position}\\
\text{Cart Velocity}\\
\text{Pole Angle}\\
\text{Pole Tip Velocity}\\
\text{Action}
\end{bmatrix}$$
...
3
votes
How to deal with categorical feature of very high cardinality?
You can use embedding which is mentioned in the comments. e.g. A general blog post, Keras documentation for embedding layer which can be used to learn the embedding. This is widely used by deep ...
- 131
3
votes
Combining Latitude/Longitude position into single feature
The best practice is to not attempt to flatten Earth into a onee dimensional line... Because as you may know, Earth more resembles a sphere than a line. It is much better to treat it as such properly.
...
- 8,114
3
votes
Finding if an outcome is predictable
That would be a part of feature selection. There are many methods to find out if there are relationships between the dependent variable and independent variables. To name a few: plots, measures of ...
- 135
2
votes
Are there any tools for feature engineering?
Scikit-learn has recently released new transformers that tackle many aspects of feature engineering. For example:
You can do multiple missing data imputation techniques with the ...
- 290
2
votes
Accepted
Classification: How to manage data sets where one data row depends on another data row
I assume that you are solving a supervised classification problem, that is, you train your model on a labeled sample. I can think of two approaches to this problem.
I. Classify tags, use neighbor ...
- 1,551
2
votes
How to combine categorical and continuous input features for neural network training
I had been using the naive structure proposed by you for quite some time by now. In a well framed problem and with enough data, this type of architecture works quite well. However here are a few ...
- 121
2
votes
how to evaluate feature quality for decision tree model
Another approach to evaluate features is called Permutation Importance. In short, this approach is random sampling values of each feature and each time measuring the negative impact this has on the ...
- 311
2
votes
Accepted
Why adding combinations of features would increase performance of linear SVM?
Multiplication is not a linear operation. Your linear SVM constructs a (hyper-)plane
$$
w_0 = w_1 x_1 + w_2 x_2
$$
for some weights $w_0, w_1, w_2.$
By introducing the AND-feature, you add another ...
- 1,646
2
votes
Creating a metric based on some features
Indeed, there are methodologies that have been tested elsewhere, some with greater and less success.
I will propose one of them to build a prediction of job satisfaction, which you can then enter as ...
- 21
2
votes
Importance of features
I'm not sure irrelevant is the right word, but here is an example where it may not be an easy trend.
You're trying to predict blood pressure.
If you're only given weight, there may be some ...
- 171
2
votes
Encode 10k features where each feature is having more than 500 categories
There is no single best technique for anything. You will have to try multiple techniques and see which one gives the best result. Also since your categorical variables are all high cardinal variables, ...
- 2,165
1
vote
Accepted
How to put multiple features into RNN input vector
A common way to input several features to an LSTM (or any RNN) is, as you did, to concatenate them in a vector. I suspect your NaN are related to a different issue in the code, and I recommend you to ...
- 641
1
vote
Accepted
I want to create an additional feature(column) based on some manipulation of values from existing features
Welcome to the community!
The code below is a starter. You can go on by naming the column and adding it to the original DataFrame:
...
- 6,712
1
vote
Using historical label as a feature in my ML model?
Correct me if I'm wrong, but it sounds like you are using an EMA to generate the price that you will be using to label as either 1,0, or -1. If so, that is not a problem to do so with historical data, ...
- 179
1
vote
Accepted
how to represent location-code as a feature in machine learning model?
You can get as creative as you want, but here are two general approaches that work for me.
Clustering the data into known geographical divisions and create dummy variables. For example, in the ...
- 2,440
1
vote
Too much inputs = overfitting?
Based on my experience so far, having too many features as inputs to your NN ,tends to degrade performance *full disclaimer i'm no expert, but smarter people than me have coined a term called The ...
- 41
Only top scored, non community-wiki answers of a minimum length are eligible