I have a dataframe that looks like this:
sentence intent hi greeting hello greeting buy this buy whats up conversation . .
What I'd like to do is take this dataframe, compute the TF-IDF and then use the TF-IDF values on a new query to calculate cosine similarity. For example, if the user types in "hi how are you?" the sentence that is most similar will be printed out with its intent.
Currently I have the TF-IDF of the dataframe:
from sklearn.feature_extraction.text import TfidfVectorizer v = TfidfVectorizer() x = v.fit_transform(intent_data["sentence"])
How can I get the TF-IDF of the new sentence and then use it to get cosine similarity to find the document (sentence) most closely related to the user typed sentence? Note: I know the dataframe example has very short sentences, but it is just used as an example.