Using HashingVectorizer for text vectorization

Question

Here is the sample data I have:

Tag 1(Val: X), Tag 2(Val: Y), Tag 3(Val: Z), Label (Val: P)

Tag 1(Val: A), Tag 2(Val: B), Tag 3(Val: C), Label (Val: Q)

Tag 1(Val: D), Tag 2(Val: E), Tag 3(Val: F), Label (Val: R)

Tag 1(Val: G), Tag 2(Val: H), Tag 3(Val: I), Label (Val: S)

I started by putting the Tags into a dataframe df and the Label into a separate dataframe df_label. Then used a HashingVectorizer to prepare the text for processing by ML models (I want to hash the strings into a unique numerical value so that the ML Models can train on it)

vectorizer = HashingVectorizer()

X_train = vectorizer.transform(df)

y_train = vectorizer.transform(df_label)

clf = RandomForestClassifier(n_jobs=2, random_state=0)

clf.fit(X_train, y_train)

When I execute this, I get: ValueError: Unknown label type: 'unknown' on y_train.

I am new to both Python and ML and I am not sure whether the problem is with my basic logic or whether it is a trivial implementation issue. Appreciate your insight and support.

Answer 1

I was recently checking some things out. Thought would leave a working code here, in-case its helpful.

from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction import FeatureHasher
from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics
import numpy as np

categories = [
    'alt.atheism',
    'talk.religion.misc',
    'comp.graphics',
    'sci.space',
]
newsgroups_train = fetch_20newsgroups(subset='train', shuffle=True, 
categories=categories, random_state=91)
newsgroups_test = fetch_20newsgroups(subset='test', shuffle=True, 
categories=categories, random_state=91)

vectorizer = FeatureHasher(input_type='string')
X_train = vectorizer.fit_transform(newsgroups_train.data)
X_test = vectorizer.fit_transform(newsgroups_test.data)

Y_train = newsgroups_train.target
Y_test = newsgroups_test.target
print(X_train.shape)
print(Y_train.shape)
print(X_test.shape)
print(Y_test.shape)

rf = RandomForestClassifier(n_jobs=-1, n_estimators=100)
rf.fit(X_train, Y_train)
pred = rf.predict(X_test)

score = metrics.accuracy_score(Y_test, pred)
print("accuracy: {:.3f}".format(score))

Answer 2

Although it is very difficult to understand your data sample i will try to correct you from what i have understood from your question.

Whenever You use any vectorizer make sure you first apply fit on your corpus/data and then transform it.

In your case you have apply transform() without applying fit on your X_train.

Important note : You don't have to perform HashingVectorizer on your Label . correct your code:

   vectorizer = HashingVectorizer()
   X_train = vectorizer.fit_transform(df)
   clf = RandomForestClassifier(n_jobs=2, random_state=0)
   clf.fit(X_train, df_label)

I would suggest to use TfidfVectorizer() instead if HashingVectorizer() but before that do some research on this.

Always refer sklearn documentation so it will help you

Hope it helps!

Answer 3

From ValueError: Unknown label type: 'unknown' on y_train I guess you have some unsupported/invalid data type in your y_train or the type is indeed valid but does not match with the RandomForestClassifier expectation.

Please do the followings:

1. print/plot the y_train and make sure all values make sense. There is a fair chance that HashingVectorizer could not hash some special cases (eg foreign language characters or missing values).
2. use type or dtype to find out the v_train data type. Make sure it is consistent with RandomForestClassifier expectation.

---

p.s. as @outlier mentioned, we usually do not transform our labels (y_train in your case). Many classifiers can handle strings/letters as class labels. In case they cannot, we can use a simple mapping such as P -> 0, Q -> 1, etc. The following does the job for you:

from sklearn.preprocessing import LabelEncoder
encoder = LabelEncoder()
encoder.fit(y)
encoded_y = encoder.transform(y)

Answer 4

You're performing a transform on your Label dataset. You dont have to do that. the vectorizer is only for the text dataset. Because youre performing a transform on your labels, it will return a matrix sparse. The fit method of your ML algorithm just asks for an Array shaped labels. Read the documentation on scikit-learn.

I hope you understand what you did wrong.