What is the difference between explainable and interpretable machine learning?

Question

O’Rourke says that explainable ML uses a black box model and explains it afterwards, whereas interpretable ML uses models that are no black boxes.

Christoph Molnar says interpretable ML refers to the degree to which a human can understand the cause of a decision (of a model). He then uses interpretable ML and explainable ML interchangably.

Wikipedia says on the topic "Explainable artificial intelligence" that it refers to AI methods and techniques such that the results of the solution can be understood by human experts. It contrasts with the concept of the black box in machine learning where even their designers cannot explain why the AI arrived at a specific decision. The technical challenge of explaining AI decisions is sometimes known as the interpretability problem.

Doshi-Velez and Kim say that that interpretable machine learning systems provide explanation for their outputs.

Obviously, there are a lot of definitions but they do not totally agree. Ultimatively, what should be explained: The results of the model, the model itself or how the model makes decissions? And what is the difference between interpret and explain?

Answer 1

I found this article by Cynthia Rudin which goes a bit more into the difference between the two terms that is in line with your source from O'Rourke.

At the core it is about the time and mechanism of the explanation:

A priori (Interpretable) vs. A posterio (Explainable)

I found this quote to be very helpful and inline with my own thoughts (emphasis mine):

Rather than trying to create models that are inherently interpretable, there has been a recent explosion of work on ‘explainable ML’, where a second (post hoc) model is created to explain the first black box model. This is problematic. Explanations are often not reliable, and can be misleading, as we discuss below. If we instead use models that are inherently interpretable, they provide their own explanations, which are faithful to what the model actually computes.

In short an interpretable model is able to output humanly understandable summaries of its calculation that allow us understand how it came to specific conclusions. Due to that a human would be able to actually create a specific desired outcome by selecting specific inputs.

A "merely" explainable model however does not deliver this input and we need a second model or mode of inspection to create a "Hypothesis about its mechanism" that will help explain the results but not allows to rebuild results by hand deterministically.

Answer 2

As for as explanation is concerned, we need explainability/interpretability at every level-

• data explanation- tsne, simple plotting.
• model explainability- by creating surrogate models
• global explainability- feature importance for all training data
• local explainability- explanation of every prediction.

all types of explainability on iris dataset, will be fun to have a look

I myself is confused about new buzz words coming everyday related to XAI( Even after developing our own xai framework) . consider Interpretability and explainability refer to same thing. people has given different names only.

Answer 3

Explainable Machine Learning is the domain of AI. It consists of interpretable models. One could say the difference is that one is a tool and the other is a field of study.

In brief, interpretable machine learning is a tool used to solve problems present in the domain of explainable machine learning.

To define your answer: One shall use an interpretable model to help " explain " the model and explain why the model gives out the specific results.

Detail Explanation: Assume you need a cnn to classify whether there is a dog in the image. The architecture of the cnn would be the interpretable aspect of the machine learning problem. And the final saliency map or heatmap which shows the output and the focus of the cnn would be the explainable part of it.

Answer 4

In my opinion, the interpretability of an ML model refers to the ability to understand how the ML model is formed. Normally, an ML model is created by using some intuitions. However, if the model is designed based on prior knowledge, like an unroll algorithm, then we know how it works. Explainability refers to the question Why, like why the model makes a decision that way. For example, a sentiment analysis model concludes a text with a positive label. Why is that? Because the text contains some indicate words like amazing, etc.

Answer 5

Interpretability can be seen as a passive chracteristic of the model that referees to which level a given model makes sense for a human observer.

Explainability can be viewed as an active characteristic of a model, denoting any action or procedure taken by a model with the intent of clarifying or detail its internal functions

Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI: https://www.sciencedirect.com/science/article/pii/S1566253519308103