My question is not about explaining the model or the algorithm

like which neurons were triggered and what are the parameters of perceptrons.

I will explain further

The problem

I have medical reports I want to analyse and extract diagnoses from it.

I built deep neural network model and extracted diagnoses with good accuracy 80%

That is good.

Now When patient look at the results and say: hmmmm, your results says I have Corona, how did the computer know?

at this moment there is no way to answer that.

The predictive model gives us the diagnoses not how it was diagnosed.


Is there any mechanism or technique that allows us to find how did the computer made this decision?

If patient was classified with diabities, the model shoud identify words that led to the identification.

Results : Diabities

Explanation : 
               the word "Thirst"
               the word "urinating frequently"
               "weight loss"

I have been googling with not much luck and I don't where to look and what is the topic name?

Any idea how to do that or at least guidance of where I can find the answer?


There are libraries like ELI5 or LIME, which can provide explanations for text classification, here is a link to an example: https://eli5.readthedocs.io/en/latest/tutorials/black-box-text-classifiers.html#textexplainer

import eli5
from eli5.lime import TextExplainer

te = TextExplainer(random_state=42)
te.fit(doc, pipe.predict_proba)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.