I created a CNN using TensorFlow2 and trained it as a binary classifier. Is there a way to extract the influence of each pixel upon the prediction?

I am trying to obtain a mask similar to the following:

husky vs wolf

  • $\begingroup$ Could you add some more details? For example your source for the example etc. $\endgroup$ – oW_ Mar 17 '20 at 2:54


You are looking for GradCam.


  1. https://fairyonice.github.io/Grad-CAM-with-keras-vis.html

  2. https://tf-explain.readthedocs.io/en/latest/index.html

Paper: https://arxiv.org/abs/1610.02391

This is a good resource I found https://christophm.github.io/interpretable-ml-book/

  • 1
    $\begingroup$ I found an existing implementation for TensorFlow called tf-explain. Thank you! $\endgroup$ – Ariana Gall Mar 18 '20 at 16:35
  • $\begingroup$ Thank you. I will update the answer with it. $\endgroup$ – Narahari B M Mar 18 '20 at 21:18

The tf-explain package supports many interpretability methods. In particular, Grad CAM can "visualize how parts of the image affects neural network's output by looking into the activation maps":

from tf_explain.callbacks.grad_cam import GradCAMCallback

model = [...]

callbacks = [
        validation_data=(x_val, y_val),

model.fit(x_train, y_train, batch_size=2, epochs=2, callbacks=callbacks)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.