Why is my test accuracy higher than train accuracy? SKLEARN

Question

As the title say my code produces low train accuracy and high test accuracy.

First I split my data set to train and test sets.

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

I then fitted my model with the training sets

model = LogisticRegression(max_iter=10000, penalty='l2')
model.fit(X_train, y_train)

After which I printed the Training accuracy of the model

predictions = model.predict(X_train)
print('Training Accuracy: {}'.format(accuracy_score(y_train, predictions)))

The training accuracy showed 0.8275. I then validated my model using stratified KFold cross validation and it gave me an 0.8125 mean accuracy score.

cross_val = StratifiedKFold(n_splits=10, random_state=1, shuffle=True)
scores = cross_val_score(model, X_train, y_train, scoring='accuracy', cv=cross_val, n_jobs=-1)

for index, score in enumerate(scores):
    print('Iteration {} Accuracy score: {}'.format(index + 1, score))
    
print('\nMean Accuracy: {}'.format(np.mean(scores)))

After that I evaluated by testing set with the following code:

testing_predictions = model.predict(X_test)

print(classification_report(y_test,testing_predictions))

cm = confusion_matrix(y_test, testing_predictions, labels=model.classes_)
display = ConfusionMatrixDisplay(confusion_matrix = cm, display_labels=model.classes_)
display.plot()

Surprisingly it gave me a higher testing accuracy of 0.86 than the training accuracy, 0.82. I have also red that it is impossible to have high testing accuracy than the training accuracy. Did I do something wrong in my process?

Answer 1

With your sample size I can see that happening by chance... rerun your train/test split 100 times (drop random seed) and see how often the performance is better on the test set.

Answer 2

It is definitely possible to have a testing accuracy that is higher than the training accuracy. If the difference between the two is relatively large you are probably underfitting your model, i.e. not using all of the signal that is in the data. In your case the difference is quite small, so I don't see anything wrong with your approach. It just means that you are likely successful in preventing your model from overfitting on your data.

Answer 3

As Oxbowerce mentioned already above. It is possible to have a test score that is higher than the training score. This is not an over-fitting scenario as well. Because your training score is lower than the test score, it is the opposite of the over-fitting case.

It might happen if test data is easier to predict than training data. So, try randomly changing or increasing your test data when you split it. You can also use shuffle=True when splitting data.