Decision Tree generating leaves for only one case

Question

I had asked a question regarding predictive analysis for marketing earlier. Prediction model for marketing to prospective customers (using pandas) Still have some doubts about it, but I have a doubt regarding the decision tree that I generated for the marketing data.

My aim is to predict if a lead will be won or lost, depending on how they were made aware of the product etc. I have a bool variable "Won", 0-sale had failed, 1-sale was made. Using a decision tree, I was able to generate a model, however, there are no leaves for cases that lead to Not Won.

Is this normal? Ive seen examples of the iris data set where all 3 features were represented in the tree, adn, hence am wondering if the approach Ive taken is correct.

In the dataset of 38000, there are roughly 1700 who have Won=1.

Im using pandas and my parameters for DecisionTreeClassifier are:

min_samples_split=2 min_samples_leaf=1 max_depth=3

I got then using grid search for parameter optimisation. I got a mean val of 0.95. If I use a bigger depth, the tree becomes too big to analyse.

Thanks

EDIT

Since Mark said to post some code, I am doing so

from sklearn.tree import DecisionTreeClassifier,export_graphviz
dt=DecisionTreeClassifier(min_samples_split=2,min_samples_leaf=1,max_depth=3)
dt.fit(x,y)
features=x.columns
#print(features)
with open("dt.dot", 'w') as f:
    export_graphviz(dt, out_file=f,feature_names=features,class_names=["Won","Lost"])
command = ["dot", "-Tpng", "dt.dot", "-o", "hello.png"]
try:
    subprocess.check_call(command)
except:
    exit("Could not run dot, ie graphviz, to produce visualization")

from sklearn.cross_validation import cross_val_score
scores=cross_val_score(dt,x,y,cv=10)
print(scores.mean())

This is the main training code, all previous lines are just munging

The cross validation score come to 0.95

Here is the csv snapshot

I use all values from "Won" onwards and am training for ""Won"

There was a single column X, which had many categorical values (20), "Won" being one of them.

Known' 'Recycled' 'Engaged' 'Prospect/MQL' 'Intern Transfer' 'MQL'
 'Working' 'Opportunity' nan 'Current Customer' 'Vendor' 'Disqualified'
 'Converted' 'SAL' 'SQL' 'engaged' 'working' 'Won' 'Web Registration'

'Inactive

However, they were all uniformly distributed ie. from 37000, all had almost the same number of observations. I used get_dummies to transform to numerical values, and dropped all the columns except "Won"

All the rest of the values value for things like designation, opp(money value scaled from 1-3) and other categories, which are all boolean

Answer 1

You are predicting binary class: 0 - sale failed, 1 - sale succeed. In the decision tree, the information value is sorted. So for example in the first node, you have 35011 predictions of 0 and 1785 predictions of class 1.

But then, in your code you have this: class_names=["Won","Lost"]). So you are telling your decision tree that name of class 0 is "Won" and name of class 1 is "Lost". I assume it should be inverted.

So in fact, the model always predict 0 - sale failed. Which seems reasonable as you said that out of 38000 samples, only 1700 are class 1.

Further, your dataset is highly unbalanced. So when cross validation gives accuracy of 95%, it says nothing, because only if your model gives all prediction 0, you will have accuracy of...95%. For those cases, use models which can put weights to classes (SVM, Logistic regression or Random Forest classifier in scikit have this possibility). Think about undersample your dataset. Or add more of class 1 - dig in your database or do it artificially (SMOTE),

Plot confusion matrix and give CV different score measure - precision or recall, depends on your situation.

Last thing, don't expect that your decision tree with depth of 3 will perform well. It is nice to show to business, but it gives poor prediction.