205 votes
Accepted

When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?

There are some cases where LabelEncoder or DictVectorizor are useful, but these are quite limited in my opinion due to ...
  • 6,738
56 votes

When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?

While AN6U5 has given a very good answer, I wanted to add a few points for future reference. When considering One Hot Encoding(OHE) and Label Encoding, we must try and understand what model you are ...
39 votes
Accepted

Ways to deal with longitude/latitude feature

Lat long coordinates have a problem that they are 2 features that represent a three dimensional space. This means that the long coordinate goes all around, which means the two most extreme values are ...
33 votes
Accepted

Encoding features like month and hour as categorial or numeric?

Have you considered adding the (sine, cosine) transformation of the time of day variable? This will ensure that the 0 and 23 hour for example are close to each other, thus allowing the cyclical nature ...
  • 466
27 votes

Encoding categorical variables using likelihood estimation

I was learning this topic too, and these are what I found: This type of encoding is called likelihood encoding, impact coding or target coding The idea is encoding your categorical variable with the ...
22 votes
Accepted

Should one hot vectors be scaled with numerical attributes

Once converted to numerical form, models don't respond differently to columns of one-hot-encoded than they do to any other numerical data. So there is a clear precedent to normalise the {0,1} values ...
  • 27.7k
20 votes

How to perform feature engineering on unknown features?

You do not need domain knowledge (the knowledge of what your data mean) in order to do feature engineering (finding more expressive ways of framing your data). As Tu N. explained, you can find "...
  • 1,346
20 votes
Accepted

Why do we convert skewed data into a normal distribution

You might want to interpret your coefficients. That is, to be able to say things like "if I increase my variable $X_1$ by 1, then, on average and all else being equal, $Y$ should increase by $\beta_1$"...
19 votes

What is difference between one hot encoding and leave one out encoding?

They are probably using "leave one out encoding" to refer to Owen Zhang's strategy. From here The encoded column is not a conventional dummy variable, but instead is the mean response over ...
18 votes

Encoding features like month and hour as categorial or numeric?

The answer depends on the kind of relationships that you want to represent between the time feature, and the target variable. If you encode time as numeric, then you are imposing certain restrictions ...
  • 631
17 votes
Accepted

Dissmissing features based on correlation with target variable

You've really got a classification problem on your hands, not a regression problem. Your target is not continuous, and Pearson correlation measures a relationship between continuous variables really. ...
  • 6,465
17 votes

Is feature engineering still useful when using XGBoost?

Feature selection: XGBoost does the feature selection up to a level. In my experience, I always do feature selection by a round of xgboost with parameters different than what I use for the final model....
  • 451
17 votes
Accepted

In ML why selecting the best variables?

You are right. If someone is using regularization correctly and doing hyperparameter tuning to avoid overfitting, then it should not be a problem theoretically (ie multi-collinearity will not reduce ...
16 votes

Is feature engineering still useful when using XGBoost?

Let's define first Feature Engineering: Feature selection Feature extraction Adding features through domain expertise XGBoost does (1) for you. XGBoost does not do (2)/(3) for you. So you still ...
  • 1,007
13 votes

Automatic Feature Engineering

In my experience, when people claim to have an automated approach to feature engineering, they really mean "feature generation", and what they're actually talking about is that they've built a deep ...
  • 1,473
11 votes

List of feature engineering techniques

Missing Data Imputation: Complete case analysis Mean / Median / Mode imputation Random Sample Imputation Replacement by Arbitrary Value Missing Value Indicator Multivariate imputation ...
  • 231
11 votes
Accepted

What feature engineering is necessary with tree based algorithms?

Feature engineering that I would consider essential for even tree based algorithms are: Modular arithmetic calculations: e.g. converting a timestamp into day of the week, or time of day. If your ...
10 votes

List of feature engineering techniques

There is no definite source on how to do feature engineering. It is often dependent on the problem you are trying to solve. Some say it is more of an art than it is science. But I would go through ...
  • 718
10 votes
Accepted

Should features be correlated or uncorrelated for classification?

Q1) Should highly correlated features with the target variable be included or removed from classification and regression problems? Is there a better/elegant explanation to this step? Actually there's ...
  • 23.8k
9 votes

Encoding categorical variables using likelihood estimation

Target encoding is now available in sklearn through the category_encoders package. Target Encoder class category_encoders.target_encoder.TargetEncoder(verbose=0, cols=None, drop_invariant=False, ...
  • 390
8 votes

Encoding features like month and hour as categorial or numeric?

I recommend using numerical features. Using categorical features essentially means that you don't consider distance between two categories as relevant (e.g. category 1 is as close to category 2 as it ...
8 votes
Accepted

What is the meaning of hand crafted features in computer vision problems?

"Hand Crafted" features refer to properties derived using various algorithms using the information present in the image itself. For example, two simple features that can be extracted from images are ...
  • 166
8 votes
Accepted

Should I rescale tfidf features?

The most accepted idea is that bag-of-words, Tf-Idf and other transformations should be left as is. According to some: Standardization of categorical variables might be not natural. Neither is ...
  • 3,310
8 votes
Accepted

Combining Latitude/Longitude position into single feature

A note: for those who've ended here looking for a hashing technique, geohash is likely your best choice. Representing latitude and longitude in a single linear scale is not possible due to the fact ...
8 votes

Why does frequency encoding work?

Check this post. In the cases where the frequency is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on ...
8 votes

Are linear models better when dealing with too many features? If so, why?

There is some important information missing in your question, i.e. what the standard parameters are and what kind of logistic regression you use. When you use ...
  • 7,054
7 votes

Using time series data from a sensor for ML

You have time series data which is used to measure the acceleration. You which to identify when the machine is in its nominal state (OFF) and anomalous state (ON). This problem would be best solved ...
  • 8,618
7 votes
Accepted

Why is duplicating inputs bad?

The issue with building a regression model on all 3 of these is that you are potentially introducing multicollinearity into the model. Although log(input) and sqrt(input) are not linear functions of ...
  • 523
7 votes

Adding feature leads to worse results

To put it shortly, xgboost tries to fix it and although it is very good in getting rid of overfitting, it is not perfect. Adding new features is not always beneficial, because you increase the ...
  • 1,480
7 votes

Is this a good practice of feature engineering?

1) Yes, it makes sense. Trying to create features manually will help the learners (i.e. models) to graspe more information from the raw data because the raw data is not always in a form that is ...
  • 71

Only top scored, non community-wiki answers of a minimum length are eligible