# Accuracy doesn't match in Keras

I am using Keras to classify images. I am following the Keras blog. The accuracy from predict_generator is not matching with the accuracy obtained from the confusion matrix, which I am computing using the scikit-learn package. I have included the relevant snippet of the code below

from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Convolution2D, MaxPooling2D
from keras.layers import Activation, Dropout, Flatten, Dense
import numpy as np
import theano
from sklearn.metrics import classification_report, confusion_matrix
y_actual = np.ones((nb_test_samples),dtype = int)
y_actual[0:2817] = 0

train_datagen = ImageDataGenerator(
featurewise_std_normalization=False,
samplewise_std_normalization=False,
rescale = 1./255)

test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_directory(
train_data_dir,
target_size = (img_width,img_height),
batch_size = 32,
class_mode = 'binary')

test_generator = test_datagen.flow_from_directory(
test_data_dir,
target_size = (img_width,img_height),
batch_size = 32,
class_mode = 'binary',
shuffle = False         )

model.fit_generator(
train_generator,
samples_per_epoch = nb_train_samples,
nb_epoch = nb_epoch,
validation_data = test_generator,
nb_val_samples = nb_test_samples)

score =   model.evaluate_generator(
test_generator,
4938)
print "Test fraction correct (Accuracy) = {:.2f}".format(score[1])
prediction = model.predict_generator(test_generator,nb_test_samples)

for i in xrange(0,len(prediction)):
if prediction[i]<0.5:
prediction[i] = 0
else:
prediction[i] = 1

#y_predicted = test_generator.classes
print np.sum(prediction)
CM = confusion_matrix(y_actual,prediction)
print CM


If I use y_predicted, I get a perfectly diagonal confusion matrix, when the console output shows an accuracy of 70% which doesn't make any sense at all. What is that I am doing wrong?

• Same problem. Did you fix this?
– user24516
Sep 20 '16 at 19:18
• @YimingYan I haven't been able to fix this. Have you been able to make any progress into this? Sep 22 '16 at 17:52
• Did you fixed it ? Kindly explain how you did. Nov 9 '17 at 10:41

The score method used in keras does not calculate accuracy like the sklearn's accuracy_score method. Check the internal implementation of evaluate method to understand more. If you want to calculate accuracy, I suggest you to use sklearn's accuracy_score by getting the predictions or manually calculate if it is easier for you.