0
$\begingroup$

I am a newbie here and trying to make sense out of the scores from model.evaluate from what I am actually seeing in model.predict

I have a created a CNN model for the Google Audio Set data and achieved a 99%+ accuracy on training.

Here is how I do the prediction

model = load_model('model_audioset.h5')
for x, y in unbal_generator:
  score = model.evaluate(x, y, verbose=0)
  pred_y = model.predict(normalized_x)

Here is what I am seeing for one specific iteration of x and y from model.evaluate

model.metrics_names = {list: 2} ['loss', 'acc']
 0 = {str} 'loss'
 1 = {str} 'acc'

score = {list: 2} [0.03851451724767685, 0.9905123114585876]
 0 = {float64} 0.03851451724767685
 1 = {float64} 0.9905123114585876

Here is a readable output from model.predict and comparing it to y

Actual vs. Expected

Wondering how Keras came up with an accuracy score of 99.05 for this output? Clearly, the predicted classes are not the same as expected


I am using a binary_crossentropy loss function and sigmoid activation in the predictions layer as classes are NOT mutually exclusive

$\endgroup$
1
$\begingroup$

Actually, I think accuracy is not a good metric for this case as y_actual and y_expected have most values 0 and the length of y_actual is quite big too.

So the accuracy calculation using an equation like K.mean(K.equal(K.argmax(y_true, axis=-1), K.argmax(y_pred, axis=-1))) (Reference) is not a good metric on model performance

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.