“Machine Learning Interpretability” or “Explainable Artificial Intelligence” has become quite popular in the machine learning community and in recent research. The goal is to make complex (deep learning) models explainable such that one can understand why the model made a particular decision. I had a look at various algorithms which do this (prominent ones like LIME, SHAP, Grad-Cam, but I've also skimmed over many papers that present very “special” approaches).

Since I am working with image data, I am particularly interested in explaining image classification decisions. However, what I realized is that, even though there is a manifold of algorithms, the (final) representation of an explanation of an image classification decision for the end user is always the same: It is the original image with some marks showing what part of the image had the most influence (whatever this means based on the underlying mathematical explanation algorithm) on the decision. Concretely, the explanations are always the original image and some sort of heatmap or outlines on top, marking the "most influential" pixels.

What other representation of an explanation of an image classification could be used? Are there any explanation methods that allow a representation as a textual or numeric explanation (instead of just heatmaps or outlines on the original image)?

  • $\begingroup$ While not textual or numeric, there is always Deep Dream (or Feature Visualization). $\endgroup$
    – Moobie
    Commented Jun 10, 2020 at 10:55
  • $\begingroup$ Thank you, @Moobie, it's actually a great idea which I'll dig deeper into. However, I am still wondering if there are any explanations that can be visualized without displaying the image (whose prediction is explained) at all. Something like "The AI predicted 'dog' in this picture, because XYZ"... Thanks for your help! $\endgroup$
    – member_10
    Commented Jun 10, 2020 at 14:27


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.