I thought that they both use one-hot encoding
These are utility to preprocess your text. Like any other utility, it has multiple options to tweak your text. You should explore all the parameters using the official docs.
I will explain one of these i.e. OHE vs Count
from sklearn.feature_extraction.text import CountVectorizer
corpus = [ 'This movie is bad.Too Bad', 'Awesome Movie. Too Awesome']
vectorizer = CountVectorizer(binary=True) #binary=False will make it Count
x = vectorizer.fit_transform(corpus)
import pandas as pd
df = pd.DataFrame(x.toarray(), columns=vectorizer.get_feature_names())
Our end goal is to create Features and each Feature has an indicator for its contextual meaning.
what I don't understand is why CountVectorizer is not used on Deep Learning models such as RNN and Tokenizer() is not used on ML Classifiers such as SVM,
When we are modeling a simple ML algorithm, we generally use
So there is no point in adding
Same is True for Deep learning.
Though, in this case, we have another reason too i.e. Deep Learning generally works on a large dataset. So, we mostly use the idea of embedding on our features. So it's better to use a Framework that provides an end-to-end solution