0
$\begingroup$

I am working on a binary classification problem with 1000 rows and 10 features.

While I did use random forest for classification, I also used LIME to explain the predictions of the random forest. However, I came across something like below

Intercept 0.7932393836062923  
Prediction_local [0.71440155]
Right: 0.6854552819361831     

Lime computes prediction_local based on below formula

exp.local_exp = exp.intercept[1] + sum([weight[1] for weight in exp.local_exp[1]])
0.714401551296631 #returned this value (matches with `Prediction_Local`)

So, my question is

a) When individual features contributes very less to the prediction, how can I convince the business regarding huge intercept value? what is the use of intercept? If business asks why is intercept value high, how can I explain the reasoning behind it?

b) I understand that intercept helps us to capture all linear patterns (which a model with no intercept cannot capture) but how does it acquire it's value? If you are asked to explain the use of intercept to model predictions, how would you explain that to a ordinary layman?

c) In above LIME explanation, we can see that major contribution of local prediction came from intercept. So, am trying to understand how does it get its value and how to interpret and translate its usefulness for business stakeholders? I understand intercept is a constant when X=0. So, what does that mean? How is it useful? As you can see in the above example, despite my input variable coefficients being very low, it still predicts the class/outcome correctly because of the high intercept value. But as you know this intercept value doesn't come from our input variables. So, how do we explain the reasoning of this intercept value with a simple explanation to business users?

This question is also raised because am having trouble in explaining intercept to business users without mathematical terms

$\endgroup$
1
  • $\begingroup$ Why should this question be closed? All my questions are related to the same topic, intercept (and they all are related). So, I would suggest you not to close this question. Moreover, I have also placed a bounty on this question to seek help form people and help me understand it in simple terms $\endgroup$
    – The Great
    Feb 12, 2022 at 6:40

1 Answer 1

3
+25
$\begingroup$

There is no intercept term in Random Forest binary classification model. The intercept term in your problem is from LIME (local interpretable model-agnostic explanations) which uses a surrogate model. It is not clear from your question what surrogate model you are using but let's assume its logistic regression.

An intercept means how likely is the positive class when all the features happen to have zero value. A large intercept value means a large chance of the positive class. Given that LIME only uses local prediction and can do what-if modeling, you can provide specific, concrete examples to the business. For a given data point, the model makes a prediction of the probability of positive class membership.

Since your original model is Random Forest, it might make more sense to show a visualization of the prediction path (not explain a separate surrogate model).

$\endgroup$
3
  • $\begingroup$ There are some real world examples here - texasgateway.org/resource/determining-meaning-intercepts $\endgroup$
    – The Great
    Feb 14, 2022 at 1:09
  • $\begingroup$ Can you help me with that ticket sales example where the objective is to earn 350$ revenue. They have two types of tickets. Adults and children tickets. I understand that if we have to sell only adults tickets, we have to sell 70 adults tickets to earn 350$ whereas if we have to sell only children tickets, we have to sell 140 tickets. I get this. If I put this to our problem how does intercept help in predicting positive or negative class? If it's negative class, what would have been the intercept value $\endgroup$
    – The Great
    Feb 14, 2022 at 1:11
  • $\begingroup$ Or we can take even gym membership example from above link. they use activation fee 49$ as intercept and monthly fees as input variable. But, not sure how does that constant help in prediction. Meaning, how does gym activation fee of 49$ can help us in predicting whether this customer will churn or extend his subscription package. Any simple explanation, I would really be grateful please $\endgroup$
    – The Great
    Feb 14, 2022 at 1:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.