I am using Keras to classify images. I am following the Keras blog. The accuracy from predict_generator is not matching with the accuracy obtained from the confusion matrix, which I am computing using the scikit-learn package. I have included the relevant snippet of the code below

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
import numpy as np
import theano
from sklearn.metrics import classification_report, confusion_matrix
y_actual = np.ones((nb_test_samples),dtype = int)
y_actual[0:2817] = 0

train_datagen = ImageDataGenerator(
     rescale = 1./255)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
                    target_size = (img_width,img_height),
                    batch_size = 32,
                    class_mode = 'binary')

test_generator = test_datagen.flow_from_directory(
                    target_size = (img_width,img_height),
                    batch_size = 32,
            class_mode = 'binary',
            shuffle = False         )

                    samples_per_epoch = nb_train_samples,
                    nb_epoch = nb_epoch,
                    validation_data = test_generator,
                    nb_val_samples = nb_test_samples)

score =   model.evaluate_generator(
print "Test fraction correct (Accuracy) = {:.2f}".format(score[1])
prediction = model.predict_generator(test_generator,nb_test_samples)

for i in xrange(0,len(prediction)):
    if prediction[i]<0.5:
       prediction[i] = 0
       prediction[i] = 1

#y_predicted = test_generator.classes
print np.sum(prediction)
CM = confusion_matrix(y_actual,prediction)
print CM 

If I use y_predicted, I get a perfectly diagonal confusion matrix, when the console output shows an accuracy of 70% which doesn't make any sense at all. What is that I am doing wrong?

  • $\begingroup$ Same problem. Did you fix this? $\endgroup$
    – user24516
    Commented Sep 20, 2016 at 19:18
  • $\begingroup$ @YimingYan I haven't been able to fix this. Have you been able to make any progress into this? $\endgroup$ Commented Sep 22, 2016 at 17:52
  • $\begingroup$ Did you fixed it ? Kindly explain how you did. $\endgroup$ Commented Nov 9, 2017 at 10:41

1 Answer 1


The score method used in keras does not calculate accuracy like the sklearn's accuracy_score method. Check the internal implementation of evaluate method to understand more. If you want to calculate accuracy, I suggest you to use sklearn's accuracy_score by getting the predictions or manually calculate if it is easier for you.

  • 1
    $\begingroup$ Thanks for pointing out the difference. Just so that I have understood it correctly, in this link, blog.keras.io/…, accuracies of ~91% are reported using VGG net. So, this might not be 91% or 90% or whatever that Keras reports? $\endgroup$ Commented Oct 13, 2016 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.