208 votes

When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?

There are some cases where LabelEncoder or DictVectorizor are useful, but these are quite limited in my opinion due to ...
AN6U5's user avatar
  • 6,798
55 votes

When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are ...
Pushkaraj Joshi's user avatar
39 votes

Ways to deal with longitude/latitude feature

Lat long coordinates have a problem that they are 2 features that represent a three dimensional space. This means that the long coordinate goes all around, which means the two most extreme values are ...
Jan van der Vegt's user avatar
33 votes

Encoding features like month and hour as categorial or numeric?

Have you considered adding the (sine, cosine) transformation of the time of day variable? This will ensure that the 0 and 23 hour for example are close to each other, thus allowing the cyclical nature ...
Pablo O's user avatar
  • 466
27 votes

Encoding categorical variables using likelihood estimation

I was learning this topic too, and these are what I found: This type of encoding is called likelihood encoding, impact coding or target coding The idea is encoding your categorical variable with the ...
alperovich's user avatar
24 votes

Should one hot vectors be scaled with numerical attributes

Once converted to numerical form, models don't respond differently to columns of one-hot-encoded than they do to any other numerical data. So there is a clear precedent to normalise the {0,1} values ...
Neil Slater's user avatar
  • 28.6k
20 votes

What is difference between one hot encoding and leave one out encoding?

They are probably using "leave one out encoding" to refer to Owen Zhang's strategy. From here The encoded column is not a conventional dummy variable, but instead is the mean response over ...
Dex Groves's user avatar
20 votes

How to perform feature engineering on unknown features?

You do not need domain knowledge (the knowledge of what your data mean) in order to do feature engineering (finding more expressive ways of framing your data). As Tu N. explained, you can find "...
Winks's user avatar
  • 1,366
20 votes

Why do we convert skewed data into a normal distribution

You might want to interpret your coefficients. That is, to be able to say things like "if I increase my variable $X_1$ by 1, then, on average and all else being equal, $Y$ should increase by $\beta_1$"...
Ricardo Magalhães Cruz's user avatar
19 votes

Is feature engineering still useful when using XGBoost?

Let's define first Feature Engineering: Feature selection Feature extraction Adding features through domain expertise XGBoost does (1) for you. XGBoost does not do (2)/(3) for you. So you still ...
FrancoSwiss's user avatar
  • 1,057
19 votes

Is feature engineering still useful when using XGBoost?

Feature selection: XGBoost does the feature selection up to a level. In my experience, I always do feature selection by a round of xgboost with parameters different than what I use for the final model....
Mortezaaa's user avatar
  • 471
18 votes

Encoding features like month and hour as categorial or numeric?

The answer depends on the kind of relationships that you want to represent between the time feature, and the target variable. If you encode time as numeric, then you are imposing certain restrictions ...
raghu's user avatar
  • 641
18 votes

In ML why selecting the best variables?

You are right. If someone is using regularization correctly and doing hyperparameter tuning to avoid overfitting, then it should not be a problem theoretically (ie multi-collinearity will not reduce ...
fractalnature's user avatar
17 votes

Dissmissing features based on correlation with target variable

You've really got a classification problem on your hands, not a regression problem. Your target is not continuous, and Pearson correlation measures a relationship between continuous variables really. ...
Sean Owen's user avatar
  • 6,595
13 votes

Automatic Feature Engineering

In my experience, when people claim to have an automated approach to feature engineering, they really mean "feature generation", and what they're actually talking about is that they've built a deep ...
Kyle.'s user avatar
  • 1,483
13 votes

List of feature engineering techniques

Missing Data Imputation: Complete case analysis Mean / Median / Mode imputation Random Sample Imputation Replacement by Arbitrary Value Missing Value Indicator Multivariate imputation ...
Sole G's user avatar
  • 271
12 votes

What feature engineering is necessary with tree based algorithms?

Feature engineering that I would consider essential for even tree based algorithms are: Modular arithmetic calculations: e.g. converting a timestamp into day of the week, or time of day. If your ...
Eumenedies's user avatar
10 votes

List of feature engineering techniques

There is no definite source on how to do feature engineering. It is often dependent on the problem you are trying to solve. Some say it is more of an art than it is science. But I would go through ...
phiver's user avatar
  • 718
9 votes

Encoding categorical variables using likelihood estimation

Target encoding is now available in sklearn through the category_encoders package. Target Encoder class category_encoders.target_encoder.TargetEncoder(verbose=0, cols=None, drop_invariant=False, ...
jeffhale's user avatar
  • 400
9 votes

Should I rescale tfidf features?

The most accepted idea is that bag-of-words, Tf-Idf and other transformations should be left as is. According to some: Standardization of categorical variables might be not natural. Neither is ...
wacax's user avatar
  • 3,390
9 votes

Combining Latitude/Longitude position into single feature

A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice. Representing latitude and longitude in a single linear scale is not possible due to the fact ...
Julio Cezar Silva's user avatar
8 votes

Encoding features like month and hour as categorial or numeric?

I recommend using numerical features. Using categorical features essentially means that you don't consider distance between two categories as relevant (e.g. category 1 is as close to category 2 as it ...
Tanguy Coatalem's user avatar
8 votes

What is the meaning of hand crafted features in computer vision problems?

"Hand Crafted" features refer to properties derived using various algorithms using the information present in the image itself. For example, two simple features that can be extracted from images are ...
chainD's user avatar
  • 166
8 votes

Why does frequency encoding work?

Check this post. In the cases where the frequency is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on ...
Piotr Rarus's user avatar
8 votes

Are linear models better when dealing with too many features? If so, why?

There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use. When you use ...
Peter's user avatar
  • 7,366
8 votes

How to Combat Data Drift

As you suggest, that situation could end up your monitoring system indicating a data drift. To evaluate this scenario, let's classify some types of data drift we could have: features drift: given ...
German C M's user avatar
  • 2,686
7 votes

Using time series data from a sensor for ML

You have time series data which is used to measure the acceleration. You which to identify when the machine is in its nominal state (OFF) and anomalous state (ON). This problem would be best solved ...
JahKnows's user avatar
  • 8,776
7 votes

Why is duplicating inputs bad?

The issue with building a regression model on all 3 of these is that you are potentially introducing multicollinearity into the model. Although log(input) and sqrt(input) are not linear functions of ...
bnorm's user avatar
  • 533
7 votes

Adding feature leads to worse results

To put it shortly, xgboost tries to fix it and although it is very good in getting rid of overfitting, it is not perfect. Adding new features is not always beneficial, because you increase the ...
aivanov's user avatar
  • 1,490
7 votes

Is this a good practice of feature engineering?

1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
Fansly's user avatar
  • 71

Only top scored, non community-wiki answers of a minimum length are eligible