# why does transform from tfidf vectorizer (sklearn) not work

I'm transforming a text in tf-idf from sklearn. I made the model:

from sklearn.feature_extraction.text import TfidfVectorizer
corpus = words
vectorizer = TfidfVectorizer(min_df = 15)
tf_idf_model = vectorizer.fit_transform(corpus)


And now I'm making vectors for different sets of words (documents), like:

word_set = ['dog', 'cat', 'foo']
v = vectorizer.transform(word_set)


But I want just one vector of these words, to compare to other documents. But when I use transform, the shape of v becomes:

<3x56492 sparse matrix of type '<class 'numpy.float64'>'
with 3 stored elements in Compressed Sparse Row format>


I want a vector with shape 1x56492, and not 3x56492.. I'm certainly missing something here. Maybe you guys have some tips?

Thank you very much in advance.

• word_set should be a list of lists(list of vectors). try putting [['dog', 'cat', 'foo']] – yoav_aaa May 1 '19 at 13:08
• @yoav_aaa I did t = [['dog', 'cat', 'foo']] and v = vectorizer.transform(t) and that gave me AttributeError: 'list' object has no attribute 'lower' – why_not May 1 '19 at 13:13

As far as I can tell it's interpreting your new word_set word_set = ['dog', 'cat', 'foo'] as three separate documents containing one word each, whereas if you did word_set = ['dog cat foo'] it would interpret this as a single new document containing those words.
What behavior are you expecting from this function? Is corpus = words a list of document strings, or a list of single words? If it's the latter, this is likely not doing what you think it is doing, and you should instead make corpus a list of document strings.