0
$\begingroup$

I am currently trying to build a text classifier and I am experimenting with different settings. Specifically, I am extracting my features with a CountVectorizer and HashingVectorizer:

from sklearn.feature_extraction.text import CountVectorizer, HashingVectorizer

# Using the count vectorizer.
count_vectorizer = CountVectorizer(lowercase=False, ngram_range=(1, 2)) 
X_train_count_vectorizer = count_vectorizer.fit_transform(X_train['text_combined'])
X_dev_count_vectorizer = count_vectorizer.transform(X_dev['text_combined'])

# Using the has vectorizer.
hash_vectorizer = HashingVectorizer(n_features=2**16,lowercase=True, ngram_range=(1, 2))
X_train_hash_vectorizer = hash_vectorizer.fit_transform(X_train['text_combined'])
X_dev_hash_vectorizer = hash_vectorizer.transform(X_dev['text_combined'])

Then I am using a LinearSVC classifier

from sklearn.svm import LinearSVC

# Testing with CountVectorizer.
clf_count = LinearSVC(random_state=0)
clf_count.fit(X_train_count_vectorizer, y_train)
y_pred = clf_count.predict(X_dev_count_vectorizer)
accuracy_score(y_dev, y_pred)

# Testing with HasingVectorizer.
clf_count = LinearSVC(random_state=0)
clf_count.fit(X_train_hash_vectorizer, y_train)
y_pred = clf_count.predict(X_dev_hash_vectorizer)
accuracy_score(y_dev, y_pred)

I obtained the following results:

Time to train Accuracy
CountVectorizer 59.9 seconds 83.97%
HashingVectorizer 6.21 seconds 84.92%

Please note that even when limiting the number of features of the CountVectorizer to 2**18, I still get slower training and inferior reults.

My questions:

  • Why is training with CountVectorizer slower even for a similar number of features?
  • What could explain the performance gain in terms of training time?
  • Any intuition on the reasons behind the accuracy gain?

For my particular case, I have also trained a TfidfVectorizer and the CountVectorizer worked a bit better. If the HashingVectorizer has such significant advantages in certain cases. I am wondering why the HashingVectorizer usage is not more widely introduced in different NLP tutorials?

$\endgroup$

1 Answer 1

1
$\begingroup$

Your lowercase setting is different for CountVectorizer and HashingVectorizer. It might have an impact.

Otherwise, they do very similar job in this case, the accuracy difference varies with the exact task but is not that huge. Disparate training speeds you observe are not related to the method itself (the feature matrix size is comparable), it's just HashingVectorizer normalizing the results by default, which is usually beneficial for SVC, resulting in much fewer iterations (check clf_count.n_iter_). Applying sklearn.preprocessing.Normalizer() to CountVectorizer results will likely make it fit equally fast.

HashingVectorizer is still faster and more memory efficient when doing the initial transform, which is nice for huge datasets. The main limitation is its transform not being invertible, which limits the interpretability of your model drastically (and even straight up unfitting for many other NLP tasks).

$\endgroup$
1
  • $\begingroup$ Thanks. Yes, I was aware of the lowercase setting. In both cases, I selected the optimal parameters. It turns out that the CountVectorizer worked better without lowercasing and the HashingVectorizer worked better with lowercasing. $\endgroup$
    – ryuzakinho
    Jul 20, 2022 at 5:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.