Use text classifier on unseen data

Question

I've trained a few models to classify between two categories of text. Logistic regression was the best. Now how can i test it on unseen data? I tried this:

def train_model():
 classifier.fit(feature_vector_train, label)
 predictions = classifier.predict(feature_vector_valid)
 joblib.dump(classifier, url+name)
...

load_model =joblib.load('my_model.pkl)
result = load_model.score('testx')

It tells me i need a y input. However, if it's new i don't have the label. WHat am i missing?

Answer 1

Welcome to the forums.

My understanding is that you're wanting to use the previously trained model to label new data points? If so, you'll be wanting to use .predict(X). From sklearn's documentation they say.

All supervised estimators in scikit-learn implement a fit(X, y) method to fit the model > and a predict(X) method that, given unlabeled observations X, returns the predicted labels y. (Source)

Another note, is that you can't pass direct strings to a model - you'll need to preprocess your data like you did for your training set. Here is a good example of building a classifier and using it to predict new points.

Let me know if you have any questions of I've misunderstood.

Answer 2

The problem is this line I think.

result = load_model.score('testx')

Purpose of score method is to compute how good the model is. This checks how close the predictions of the model were to the actual values of the predicted variable. That is why you need to provide the score function with the values of target variable y.

If you want to predict values, you use predict method as pointed out by @James C.

Answer 3

While performing prediction on any unseen data you have to keep in mind the following points.

1. Convert your sentences into an array like

testx = "i like that movie"
testx = np.array(testx)

2. Then perform preprocessing if you applied like removing stopwords, creating ngrams
3. Now convert that single sentence array into features using methods like countvectorization & td-idf which ever you have used while training. Make sure you have saved that model also. Load it in the prediction phase

load_cv = joblib.load('cv_model.pkl')
testx = load_cv.transform(testx)

4. Once all these steps are done then load your model like

load_model = joblib.load('my_model.pkl')
result = load_model.predict(testx)