208
votes
Accepted
When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?
There are some cases where LabelEncoder or DictVectorizor are useful, but these are quite limited in my opinion due to ...
- 6,798
55
votes
When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?
While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are ...
- 731
39
votes
Accepted
Ways to deal with longitude/latitude feature
Lat long coordinates have a problem that they are 2 features that represent a three dimensional space. This means that the long coordinate goes all around, which means the two most extreme values are ...
- 9,348
33
votes
Accepted
Encoding features like month and hour as categorial or numeric?
Have you considered adding the (sine, cosine) transformation of the time of day variable? This will ensure that the 0 and 23 hour for example are close to each other, thus allowing the cyclical nature ...
- 466
27
votes
Encoding categorical variables using likelihood estimation
I was learning this topic too, and these are what I found:
This type of encoding is called likelihood encoding, impact coding or target coding
The idea is encoding your categorical variable with the ...
- 371
24
votes
Accepted
Should one hot vectors be scaled with numerical attributes
Once converted to numerical form, models don't respond differently to columns of one-hot-encoded than they do to any other numerical data. So there is a clear precedent to normalise the {0,1} values ...
- 28.6k
20
votes
What is difference between one hot encoding and leave one out encoding?
They are probably using "leave one out encoding" to refer to Owen Zhang's strategy.
From here
The encoded column is not a conventional dummy variable, but instead is the mean response over ...
- 386
20
votes
How to perform feature engineering on unknown features?
You do not need domain knowledge (the knowledge of what your data mean) in order to do feature engineering (finding more expressive ways of framing your data).
As Tu N. explained, you can find "...
- 1,366
20
votes
Accepted
Why do we convert skewed data into a normal distribution
You might want to interpret your coefficients. That is, to be able to say things like "if I increase my variable $X_1$ by 1, then, on average and all else being equal, $Y$ should increase by $\beta_1$"...
- 3,410
19
votes
Is feature engineering still useful when using XGBoost?
Let's define first Feature Engineering:
Feature selection
Feature extraction
Adding features through domain expertise
XGBoost does (1) for you.
XGBoost does not do (2)/(3) for you.
So you still ...
- 1,057
19
votes
Is feature engineering still useful when using XGBoost?
Feature selection: XGBoost does the feature selection up to a level. In my experience, I always do feature selection by a round of xgboost with parameters different than what I use for the final model....
- 471
18
votes
Encoding features like month and hour as categorial or numeric?
The answer depends on the kind of relationships that you want to represent between the time feature, and the target variable.
If you encode time as numeric, then you are imposing certain restrictions ...
- 641
18
votes
Accepted
In ML why selecting the best variables?
You are right. If someone is using regularization correctly and doing hyperparameter tuning to avoid overfitting, then it should not be a problem theoretically (ie multi-collinearity will not reduce ...
- 805
17
votes
Accepted
Dissmissing features based on correlation with target variable
You've really got a classification problem on your hands, not a regression problem. Your target is not continuous, and Pearson correlation measures a relationship between continuous variables really. ...
- 6,595
13
votes
Automatic Feature Engineering
In my experience, when people claim to have an automated approach to feature engineering, they really mean "feature generation", and what they're actually talking about is that they've built a deep ...
- 1,483
13
votes
List of feature engineering techniques
Missing Data Imputation:
Complete case analysis
Mean / Median / Mode imputation
Random Sample Imputation
Replacement by Arbitrary Value
Missing Value Indicator
Multivariate imputation
...
- 271
12
votes
Accepted
What feature engineering is necessary with tree based algorithms?
Feature engineering that I would consider essential for even tree based algorithms are:
Modular arithmetic calculations: e.g. converting a timestamp into day of the week, or time of day. If your ...
- 441
10
votes
List of feature engineering techniques
There is no definite source on how to do feature engineering. It is often dependent on the problem you are trying to solve. Some say it is more of an art than it is science.
But I would go through ...
- 718
9
votes
Encoding categorical variables using likelihood estimation
Target encoding is now available in sklearn through the category_encoders package.
Target Encoder
class category_encoders.target_encoder.TargetEncoder(verbose=0, cols=None, drop_invariant=False, ...
- 400
9
votes
Accepted
Should I rescale tfidf features?
The most accepted idea is that bag-of-words, Tf-Idf and other transformations should be left as is.
According to some: Standardization of categorical variables might be not natural. Neither is ...
- 3,390
9
votes
Accepted
Combining Latitude/Longitude position into single feature
A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice.
Representing latitude and longitude in a single linear scale is not possible due to the fact ...
8
votes
Encoding features like month and hour as categorial or numeric?
I recommend using numerical features. Using categorical features essentially means that you don't consider distance between two categories as relevant (e.g. category 1 is as close to category 2 as it ...
8
votes
Accepted
What is the meaning of hand crafted features in computer vision problems?
"Hand Crafted" features refer to properties derived using various algorithms using the information present in the image itself. For example, two simple features that can be extracted from images are ...
- 166
8
votes
Why does frequency encoding work?
Check this post.
In the cases where the frequency is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on ...
- 814
8
votes
Are linear models better when dealing with too many features? If so, why?
There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use.
When you use ...
- 7,366
8
votes
Accepted
How to Combat Data Drift
As you suggest, that situation could end up your monitoring system indicating a data drift. To evaluate this scenario, let's classify some types of data drift we could have:
features drift: given ...
- 2,686
7
votes
Using time series data from a sensor for ML
You have time series data which is used to measure the acceleration. You which to identify when the machine is in its nominal state (OFF) and anomalous state (ON). This problem would be best solved ...
- 8,776
7
votes
Accepted
Why is duplicating inputs bad?
The issue with building a regression model on all 3 of these is that you are potentially introducing multicollinearity into the model. Although log(input) and sqrt(input) are not linear functions of ...
- 533
7
votes
Adding feature leads to worse results
To put it shortly, xgboost tries to fix it and although it is very good in getting rid of overfitting, it is not perfect.
Adding new features is not always beneficial, because you increase the ...
- 1,490
7
votes
Is this a good practice of feature engineering?
1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
- 71
Only top scored, non community-wiki answers of a minimum length are eligible