# Understanding what is going on

I have a collection of 1000s of bottles of wine. I want to understand what could / likely to be driving the price either up or down. Below is an example of the data

Wine      Country       Area        Grape        Class    Year    Price
A         France        Burgundy    Pinot Noir   Grand    2014    +6%
B         France        Burgundy    Chardonnay   1er      2014    -1%
C         France        Burgundy    Pinot Noir   1er      2014    +4%
D         USA           California  Pinot Noir            2013    +1%
E         USA           California  Blend                 2014    +0.5%
F         USA           California  Chardonnay            2014    -4%
G         USA           California  Chardonnay            2013    -5%


From these fictitious numbers i would assume the reason bottle A is up is because pinot is up but more importantly France is up V USA wine. To the same extent Chardonnay is doing poorly but USA again is doing worse than France.

Here this is rather easy but if this was extended by 1000s of rows with more categories added, the exercise becomes less easy to see the trend.

My original thought would be to use the importance of a feature from a decision tree. But is there a better way?

Thanks

What you are looking for is feature contributions to the final score of an observation having a positive or negative price. Feature importances from a decision tree or random forest is not going to help you because a feature's importance is fixed across all observations (wines). They tell a story about the overall model, but nothing about the individual observations. So if you want to know what caused a wine having a negative or positive price, you should look at feature contributions.

Let's elaborate a little bit.

Let's forget about decision trees, and go for ensemble of them, say random forest, for all the good reasons. (I highly recommend you to read why ensemble models uniformly beat the base learner, and how they help with the high bias - high variance problem base learners might have.)

The first step would be to label your dependent variable, price, as $1$ and $0$, depending on the price being positive and negative ($y = 1$ iff $price > 0$). After the typical exploratory analysis (where you can look at boxplots and stuff as a beginning study) and feature engineering, you can fit your random forest to predict a wine having a positive or negative price (i.e. $y$ being $1$ or $0$).

Once you do that, you will have an estimated probability for each and every wine, representing the classification model's estimation of that wine's price being positive or negative. Something like this:

Wine Prob
A    0.4
B    0.7
C    0.9
D    0.3


These probabilities actually consist of the summation of a bias term and the individual contributions of all features to that observation only:

$$P(y_i = 1) = bias + \sum_{i=1}^{m}(contributionOfFeature_i),$$

where $m$ is the total number of features. This is how you can analyze each and every feature's contribution on the final probability of a particulat wine's price being positive or negative.

Regarding how to code this, if you are using scikit, you can use this very easy and convenient tool:

https://github.com/andosa/treeinterpreter

• That's great and exactly what i am looking for thanks. Looks relatively simple as well. Commented Dec 11, 2017 at 10:01
• You are welcome, @ben121. Glad to hear that it helps. My job consists of giving customers specific reasons why they scored the way that did on the ML model that I built, and this is the logic that I implement. Your question was equivalent to my problem, and hence I thought it could help. Thanks! Commented Dec 11, 2017 at 16:28
• Would you suggest something like this for a situation where i already know if the price is up or down but want to know the likely causes? For me its not always about predicting but having an educated guess as to why. Commented Dec 12, 2017 at 17:30
• Absolutely! You HAVE TO know if the price is up or down to form your dependent variable in the first place. The only way to make an educated guess is to make an estimation. That way you'll find out to which features you can attribute the price being up or down. This is a typical and fully-regarded econometric approach. You estimate to which variables you can attribute the changes in the dependent variable, and the magnitudes. If the "goodness of fit" (AUC) is good, that means you really discovered the likely causes, i.e. which variables are driving the price being up or down, and how much. Commented Dec 13, 2017 at 6:18

Looks like all of your features are categorical. Just for a quick look. Try box blots of price for each variable by each category-class. If for some variable you start seeing a clear differences in the boxes (Hight of the box and the whiskers). That will help you claim that feature's importance analytically. Else doing a decision-tree and looking at feature importance seems like a good solution.