# How to extract the reason behind a prediction using TensorFlow?

I created a CNN using TensorFlow2 and trained it as a binary classifier. Is there a way to extract the influence of each pixel upon the prediction?

I am trying to obtain a mask similar to the following:

• Could you add some more details? For example your source for the example etc. – oW_ Mar 17 '20 at 2:54

[updated]

Code:

This is a good resource I found https://christophm.github.io/interpretable-ml-book/

• I found an existing implementation for TensorFlow called tf-explain. Thank you! – Ariana Gall Mar 18 '20 at 16:35
• Thank you. I will update the answer with it. – Narahari B M Mar 18 '20 at 21:18

The tf-explain package supports many interpretability methods. In particular, Grad CAM can "visualize how parts of the image affects neural network's output by looking into the activation maps":

from tf_explain.callbacks.grad_cam import GradCAMCallback

model = [...]

callbacks = [