1
$\begingroup$

I am practicing to use sklearn for decision tree, and I am using the play tennis data set:

DataSet

play_ is the target column.

as per my pen and paper calculation of entropy and Information Gain, the root node should be outlook_ column because it has the highest entropy.

But somehow, my current decision tree has humidity as the root node, and look likes this:

Decision Tree Current Scenario

My current code in python:

from sklearn.cross_validation import train_test_split 
from sklearn.tree import DecisionTreeClassifier 
from sklearn.metrics import accuracy_score 
from sklearn import tree 
from sklearn.preprocessing import LabelEncoder

import pandas as pd 
import numpy as np 

df = pd.read_csv('playTennis.csv') 

lb = LabelEncoder() 
df['outlook_'] = lb.fit_transform(df['outlook']) 
df['temp_'] = lb.fit_transform(df['temp'] ) 
df['humidity_'] = lb.fit_transform(df['humidity'] ) 
df['windy_'] = lb.fit_transform(df['windy'] )   
df['play_'] = lb.fit_transform(df['play'] ) 
X = df.iloc[:,5:9] 
Y = df.iloc[:,9]

X_train, X_test , y_train,y_test = train_test_split(X, Y, test_size = 0.3, random_state = 100) 

clf_entropy = DecisionTreeClassifier(criterion='entropy')
clf_entropy.fit(X_train.astype(int),y_train.astype(int)) 
y_pred_en = clf_entropy.predict(X_test)

print("Accuracy is :{0}".format(accuracy_score(y_test.astype(int),y_pred_en) * 100))
$\endgroup$
  • $\begingroup$ Why don't you just let the decision tree algorithm find the best nodes and thresholds automatically. Isn't that the whole purpose of using machine learning, to automate it. You clearly know what is going on under the hood. If you want to do it manually, then just code it with IF statements. $\endgroup$ – JahKnows Jan 15 '18 at 3:59
2
$\begingroup$

If you would skip this line:

X_train, X_test , y_train,y_test = train_test_split(X, Y, test_size = 0.3, random_state = 100)

and change the method parameters to train your model like this:

clf_entropy.fit(X, Y)

it should work as expected.

$\endgroup$
1
$\begingroup$

In the code, you have done a split of the data into train/test. If you have used all samples to "train" manually your decision tree you have more samples to do the calculations than the sklearn algorithm, so the results may change.

You can look which samples has been chosen to train the tree and do the calculations with those samples.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.