# Scikit Learn OneHotEncoded Features causing error in classifier

I’m trying to prepare data for input to a Decision Tree and Multinomial Naïve Bayes Classifier.

This is what my data looks like (pandas dataframe)

Label  Feat1  Feat2  Feat3  Feat4

0        1     3       2      1
1        0     1       1      2
2        2     2       1      1
3        3     3       2      3


I have split the data into dataLabel and dataFeatures. Prepared dataLabel using dataLabel.ravel()

I need to discretize features so the classifiers treat them as being categorical not numerical.

I’m trying to do this using OneHotEncoder

enc = OneHotEncoder()

enc.fit(dataFeatures)
chk = enc.transform(dataFeatures)
from sklearn.naive_bayes import MultinomialNB

mnb = MultinomialNB()

from sklearn import metrics
from sklearn.cross_validation import cross_val_score
scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')


I get this error - bad input shape (64, 16)

This is the shape of label and input

dataLabel.shape = 72 chk.shape = 72,16

Why won't the classifier accept the onehotencoded features?

EDIT adding how i got dataFeatures

dataFeatures = data[['Accpred', 'Gyrpred', 'Barpred', 'altpred']]

Y = dataLabel.ravel()

• how did you get the dataFeatures? Did you combine all the features(Feat1, Feat2..) into a list or so? and what is Y? – Hima Varsha Jul 26 '16 at 8:46
• @HimaVarsha I've edited the question to include that :) – gbhrea Jul 26 '16 at 8:48
• What does the Y mean? – Hima Varsha Jul 26 '16 at 8:55
• @HimaVarsha Y is dataLabel - I've also added that to the question – gbhrea Jul 26 '16 at 9:03

scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')


You have your Y and chk switched. That's it. :)

The signature of cross_val_score is sklearn.cross_validation.cross_val_score(estimator, X, y).

X is a matrix and y is a 1D vector with your class labels.

unlike in R, most (or all?) sklearn models do not support categorical variables. Most of the time, encoding your feature matrix X into what is called one-hot encoding is good enough.

Notice that, in some models, this hack is not the same as true native categorical support, and the performance of the model will be worse.

Invert One-Hot Encoding

Sklearn does not seem to have an easy method to invert the one-hot encoding.

It is not trivial how to do this. I found this suggestion:

def inverse(enc, out, shape):
return np.array([enc.active_features_[col] for col in out.sorted_indices().indices]).reshape(shape) - enc.feature_indices_[:-1]


Example:

import numpy as np
from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder()
X = np.array([[0, 0, 3], [1, 1, 0], [0, 2, 1], [1, 0, 2]]))
Z = enc.fit_transform(X)
print(inverse(enc, Z, X.shape))
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]
print(X)
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]


Notice:

• This only works when HotOneEncoding(sparse=True) (default) because it uses scipy sparse matrix methods (this could be changed by making the code only use numpy methods), but this is probably what you want since working with a dense matrix will kill your memory anyhow
• I think this will only work if your variables are within the range [0,something] because you lose that information in the transformation (no work-around for this other than you using something like DictVectorizer which offers you more control over the transformation.
• Thanks Ricardo, I have been stuck on this for longer than I care to admit :) Is there any way to map encoded labels back to original labels(integers) after prediction? – gbhrea Jul 26 '16 at 13:56
• @gbhrea I am a little confused. But I edited my answer. Please see if it helps... – Ricardo Cruz Jul 26 '16 at 14:44
• Ricardo - sorry I confused myself there - what I meant to say was is there any way to map encoded features back to their original labels? – gbhrea Jul 26 '16 at 14:57
• @gbhrea Ah okay, see my updated answer then :) – Ricardo Cruz Jul 26 '16 at 15:30