# How to consume single piece of text for classification in a model?

Here's a high-level blueprint of my model -

Input data::

CATEGORY    SERVICE     TITLE                               DEPARTMENT

apple       fruits      i love eating fruits.               fruitshop
mango       fruits      mangoes are yellow in color.        fruitshop
cycle       vehicle     that cycle is really expensive.     motorshop


I convert this to a sparse matrix with Tf-Idf scores, where the columns are the title columns tokenized.

I am using LinearSVC and StratifiedKFold (n_splits = 10, random_state=777, shuffle=True) able to achieve a prediction accuracy of 73%.

Got 2 doubts -

1) What happens when a string like "fresh fish" is used for classification? Because the words "fresh" and "fish" are never used before, which department will this be classified into?

2) How can I consume this string "fresh fish" in my model?

Currently, I have something like this -

...
# X.shape = (1181, 1930)
# y.shape = (1181,)
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=777)
...
pr = mod.predict(X_test)
print(pd.DataFrame(confusion_matrix(y_test, pr)))
...


I tried giving input as print(mod.predict([["fresh fish"]])) but got an error -

ValueError: X has 1 features per sample; expecting 1930


You have to repeat the same preprocessing steps for prediction as you did during training. Let's say you trained like this:

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=777)

vectorizer = TfidfVectorizer()
X_train = vectorizer.fit_transform(X_train)

mod = LinearSVC()
mod.fit(X_train, y_train)


Then you need to use that fit TfidfVectorizer to transform your test prediction as well (which you seem to have managed). Like this:

X_test = vectorizer.transform(X_test)
pr = mod.predict(X_test)


And the same goes if you just want to feed in a manual text:

X_manual = vectorizer.transform(['fresh fish'])
pr = mod.predict(X_manual)


Sklearn pipeline can also be used, this will avoid you writing preprocessing step again.

from sklearn.feature_extraction.text import TfidfVectorizer
import pandas as pd
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC

a= [(' i love eating fruits','fruitshop'),
('mangoes are yellow in color','fruitshop'),
('that cycle is really expensive','motorshop')]

df = pd.DataFrame(a, columns = ['title', 'Department'])

model = Pipeline([('tfidf', TfidfVectorizer()),
('logreg',LinearSVC())])

model.fit(df['title'],df['Department'])

model.predict(['fresh fish'])
#o/p
array(['fruitshop'], dtype=object)