1
$\begingroup$

I’m trying to prepare data for input to a Decision Tree and Multinomial Naïve Bayes Classifier.

This is what my data looks like (pandas dataframe):

Label  Feat1  Feat2  Feat3  Feat4

0        1     3       2      1
1        0     1       1      2
2        2     2       1      1
3        3     3       2      3

I have split the data into dataLabel and dataFeatures. Prepared dataLabel using dataLabel.ravel()

I need to discretize features so the classifiers treat them as being categorical not numerical.

I’m trying to do this using OneHotEncoder:

enc = OneHotEncoder()

enc.fit(dataFeatures)
chk = enc.transform(dataFeatures)
from sklearn.naive_bayes import MultinomialNB

mnb = MultinomialNB()

from sklearn import metrics
from sklearn.cross_validation import cross_val_score
scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')

I get this error: bad input shape (64, 16)

This is the shape of label and input:

dataLabel.shape = 72 chk.shape = 72,16

Why won't the classifier accept the onehotencoded features?

EDIT: Adding how I got dataFeatures

dataFeatures = data[['Accpred', 'Gyrpred', 'Barpred', 'altpred']]

Y = dataLabel.ravel()

$\endgroup$
4
  • $\begingroup$ how did you get the dataFeatures? Did you combine all the features(Feat1, Feat2..) into a list or so? and what is Y? $\endgroup$ Jul 26, 2016 at 8:46
  • $\begingroup$ @HimaVarsha I've edited the question to include that :) $\endgroup$
    – gbhrea
    Jul 26, 2016 at 8:48
  • $\begingroup$ What does the Y mean? $\endgroup$ Jul 26, 2016 at 8:55
  • $\begingroup$ @HimaVarsha Y is dataLabel - I've also added that to the question $\endgroup$
    – gbhrea
    Jul 26, 2016 at 9:03

1 Answer 1

1
$\begingroup$
scores = cross_val_score(mnb, Y, chk, cv=10, scoring='accuracy')

You have your Y and chk switched. That's it. :)

The signature of cross_val_score is sklearn.cross_validation.cross_val_score(estimator, X, y).

X is a matrix and y is a 1D vector with your class labels.

unlike in R, most (or all?) sklearn models do not support categorical variables. Most of the time, encoding your feature matrix X into what is called one-hot encoding is good enough.

Notice that, in some models, this hack is not the same as true native categorical support, and the performance of the model will be worse.

Invert One-Hot Encoding

Sklearn does not seem to have an easy method to invert the one-hot encoding.

It is not trivial how to do this. I found this suggestion:

def inverse(enc, out, shape):
    return np.array([enc.active_features_[col] for col in out.sorted_indices().indices]).reshape(shape) - enc.feature_indices_[:-1]

Example:

import numpy as np
from sklearn.preprocessing import OneHotEncoder
enc = OneHotEncoder()
X = np.array([[0, 0, 3], [1, 1, 0], [0, 2, 1], [1, 0, 2]]))
Z = enc.fit_transform(X)
print(inverse(enc, Z, X.shape))
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]
print(X)
# [[0 0 3]
#  [1 1 0]
#  [0 2 1]
#  [1 0 2]]

Notice:

  • This only works when HotOneEncoding(sparse=True) (default) because it uses scipy sparse matrix methods (this could be changed by making the code only use numpy methods), but this is probably what you want since working with a dense matrix will kill your memory anyhow
  • I think this will only work if your variables are within the range [0,something] because you lose that information in the transformation (no work-around for this other than you using something like DictVectorizer which offers you more control over the transformation.
$\endgroup$
4
  • $\begingroup$ Thanks Ricardo, I have been stuck on this for longer than I care to admit :) Is there any way to map encoded labels back to original labels(integers) after prediction? $\endgroup$
    – gbhrea
    Jul 26, 2016 at 13:56
  • $\begingroup$ @gbhrea I am a little confused. But I edited my answer. Please see if it helps... $\endgroup$ Jul 26, 2016 at 14:44
  • $\begingroup$ Ricardo - sorry I confused myself there - what I meant to say was is there any way to map encoded features back to their original labels? $\endgroup$
    – gbhrea
    Jul 26, 2016 at 14:57
  • $\begingroup$ @gbhrea Ah okay, see my updated answer then :) $\endgroup$ Jul 26, 2016 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.