9
$\begingroup$

O’Rourke says that explainable ML uses a black box model and explains it afterwards, whereas interpretable ML uses models that are no black boxes.

Christoph Molnar says interpretable ML refers to the degree to which a human can understand the cause of a decision (of a model). He then uses interpretable ML and explainable ML interchangably.

Wikipedia says on the topic "Explainable artificial intelligence" that it refers to AI methods and techniques such that the results of the solution can be understood by human experts. It contrasts with the concept of the black box in machine learning where even their designers cannot explain why the AI arrived at a specific decision. The technical challenge of explaining AI decisions is sometimes known as the interpretability problem.

Doshi-Velez and Kim say that that interpretable machine learning systems provide explanation for their outputs.

Obviously, there are a lot of definitions but they do not totally agree. Ultimatively, what should be explained: The results of the model, the model itself or how the model makes decissions? And what is the difference between interpret and explain?

$\endgroup$
4
$\begingroup$

I found this article by Cynthia Rudin which goes a bit more into the difference between the two terms that is in line with your source from O'Rourke.

At the core it is about the time and mechanism of the explanation:

A priori (Interpretable) vs. A posterio (Explainable)

I found this quote to be very helpful and inline with my own thoughts (emphasis mine):

Rather than trying to create models that are inherently interpretable, there has been a recent explosion of work on ‘explainable ML’, where a second (post hoc) model is created to explain the first black box model. This is problematic. Explanations are often not reliable, and can be misleading, as we discuss below. If we instead use models that are inherently interpretable, they provide their own explanations, which are faithful to what the model actually computes.

In short an interpretable model is able to output humanly understandable summaries of its calculation that allow us understand how it came to specific conclusions. Due to that a human would be able to actually create a specific desired outcome by selecting specific inputs.

A "merely" explainable model however does not deliver this input and we need a second model or mode of inspection to create a "Hypothesis about its mechanism" that will help explain the results but not allows to rebuild results by hand deterministically.

| improve this answer | |
$\endgroup$
1
$\begingroup$

As for as explanation is concerned, we need explainability/interpretability at every level-

  • data explanation- tsne, simple plotting.
  • model explainability- by creating surrogate models
  • global explainability- feature importance for all training data
  • local explainability- explanation of every prediction.

all types of explainability on iris dataset, will be fun to have a look

I myself is confused about new buzz words coming everyday related to XAI( Even after developing our own xai framework) . consider Interpretability and explainability refer to same thing. people has given different names only.

| improve this answer | |
$\endgroup$
1
$\begingroup$

Explainable Machine Learning is the domain of AI. It consists of interpretable models. One could say the difference is that one is a tool and the other is a field of study.

In brief, interpretable machine learning is a tool used to solve problems present in the domain of explainable machine learning.

To define your answer: One shall use an interpretable model to help " explain " the model and explain why the model gives out the specific results.

Detail Explanation: Assume you need a cnn to classify whether there is a dog in the image. The architecture of the cnn would be the interpretable aspect of the machine learning problem. And the final saliency map or heatmap which shows the output and the focus of the cnn would be the explainable part of it.

| improve this answer | |
$\endgroup$
0
$\begingroup$

In my opinion, the interpretability of an ML model refers to the ability to understand how the ML model is formed. Normally, an ML model is created by using some intuitions. However, if the model is designed based on prior knowledge, like an unroll algorithm, then we know how it works. Explainability refers to the question Why, like why the model makes a decision that way. For example, a sentiment analysis model concludes a text with a positive label. Why is that? Because the text contains some indicate words like amazing, etc.

| improve this answer | |
$\endgroup$
0
$\begingroup$

From Tim Miller - Explanation in Artificial Intelligence: Insights from the Social Sciences:

that explanation is post-hoc interpretability ... definition of interpretability of a model as the degree to which an observer can understand the cause of a decision. Explanation is thus one mode in which an observer may obtain understanding ... such as making decisions that are inherently easier to understand or via introspection. I equate ‘interpretability’ with ‘explainability’.

From another explanation from Miller:

Interpretability is the degree to which a human can understand the cause of a decision. Another one is: Interpretability is the degree to which a human can consistently predict the model's result

There are several definitions but in current xAI literature it seems to be being used interchangeably

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.