This post attempts to explain the difference between explainability and interpretability of ML models. However, the explanation is somewhat unclear. Can somebody provide specific examples of models that are explainable but not interpretable (or the over way round)?
- Explainable ML – using a black box and explaining it afterwards.
- Interpretable ML – using a model that is not black box.
Accordingly, a decision tree, for example, is interpretable since it inherently makes its decision explicit through the nodes/split points. And according to above definitions it is not explainable since it is not a black box model (interpretable models are not a subset of explainable models according to the definitions).
In contrast, a CNN, for example, is a black box model which implicitly encodes its decision making procedure. However, ex-post analysis is an approach to make such a model explainable. You can, for example, assess the feature map activations per layer to do so, as done in this article:
This analysis reveals, for example, that layer 2 gets activates by patterns such as edges and layer 3 detects more complex patterns. Obviously this ex-post analysis has a different quality than the explicitly encoded rules of a decision tree.
(Somewhat contradicting the given definitions you could say that explainable models require a larger degree of interpretation while interpretable models explain their decisions inherently - but that is only my wording and not how the authors of above articles phrase it.)