How would you describe the trade-off between model interpretability and model prediction power in layman's terms?

Question

I know it depends on the data and question asked but imagine a scenario that for a given dataset you could either go for a fairly complex nonlinear model (hard to interpret though) giving you a better prediction power perhaps because the model may see the nonlinearities present in the data, or have a simple model (perhaps a linear model or something) with less prediction power but easier to interpret. Here is a very good post discussing ideas on how to interpret machine learning models.

Industries, while being very cautious, are slowly becoming more interested in adopting more complex models! Still they want to know the trade-off clearly? A data scientist perhaps is the one sitting between data team and decision-makers, and often need to be able to explain these stuffs in layman's terms.

I am trying to brainstorm here to see what analogy you would come up with to describe such trade-off to a non-technical person?

Answer 1

Interesting question. I think that you can illustrate this by thinking about different use cases. The one example I've heard that I like is around lending decisions for loan applications. That's an algorithm but, because of regulations, it can't be strictly "black box". The decision has to be, effectively, interpretable because the bank has to give you a reason for decline on the loan. So, there's certainly better algos out there for loans that can give a binary result, but do you want a bank to just tell you yes or no?

Answer 2

Another question you can ask yourself is whether you have a business goal to understand a system in such a way that gives you information about the inputs and their relationship and how changes to those inputs impact your outcome (prediction).

A recent example of a problem I worked on that falls into this case is predicting the number of market leads by month (week, day) using spend by channel (TV, radio, digital). Here the goal wasn't just to predict how many leads would be generated given spend but also to have a framework to use to optimize lead generation around spend distribution (i.e., what is the most cost effective distribution of spend across TV, radio, and digital to generate the largest number of leads). Because of this business requirement, a neural network or SVM would not have met our goals because, while they would have provided a prediction of lead generation, they would not have provided the understanding of the inputs (spend by channel).