0
$\begingroup$

Vectorization techniques like TF-IDF are very common techniques for transforming text data into numerical data that can be more easily feed to ML Algorithms. Before we first train the model, each word of each document is given a number (a frequency) which depends on the whole data.

How can I input to the already trained model, a new custom, unseen before sentence since the model was trained with the whole dataset vectorized?

For instance, my new sentence has a different number of features compared to the dataset used for training and testing. Moreover, the frequencies, as far as I understand are computed based on all the words in the dataset, so ,from my perspective, I also need to include this dataset in the operation of vectorization of the new custom sentence.

$\endgroup$

1 Answer 1

1
$\begingroup$

I found the issue, I should use tf.transform, not transform_fit

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.