0
$\begingroup$

I've been using Scikit-Learn's OneHotEncoder to turn categorical data into binary columns, however, it seems that fitting OneHotEncoder to a dataset with numerical and categorical variables causes it to make binary columns for the numerical data too.

I've tried searching the documentation for an explicit answer, but I can't find one. Does OneHotEncoder automatically avoid encoding numerical columns? If not, how can I make a pipeline with it without splitting and re-joining the dataframe?

$\endgroup$

2 Answers 2

1
$\begingroup$

(My basic answer, post a more informed one and I'll accept it.)

Examples from this article show that the OneHotEncoder will encode for every unique value in a column.

You can check what's being encoded by using the OneHotEncoder().categories_ attribute. This attribute will give you a series of arrays that include all the unique values of each column that have been encoded for. If you feed numerical values into the OneHotEncoder you'll notice that these arrays contain every unique numerical value as a category.

To avoid this, you should pre-select your categorical columns and feed those to the OneHotEncoder. See the following SciKit-Learn Tutorial, and the reference for ColumnTransformer to see how this can be included in a pipeline.

$\endgroup$
0
$\begingroup$

Scikit-learn's OneHotEncoder will encode all variables in the dataframe by default.

If you want to encode just a subset, you need to wrap the OneHotEncoder with the ColumnTransformer.

Alternatively, you can use Feature-engine's OneHotEncoder which encodes only variables of type object or categorical by default, leaving the numerical variables the way they are.

See for example this code snippet:

import pandas as pd
from feature_engine.encoding import OneHotEncoder
X = pd.DataFrame(dict(x1 = [1,2,3,4], x2 = ["a", "a", "b", "c"]))
ohe = OneHotEncoder()
ohe.fit(X)
ohe.transform(X)

The result of that transformation is the following dataframe, where the numerical variable remains unchanged and the categorical one was one hot encoded:

   x1  x2_a  x2_b  x2_c
0   1     1     0     0
1   2     1     0     0
2   3     0     1     0
3   4     0     0     1

I leave a link to Feature-engine's OneHotEncoder for more details.

$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.