# (Newbie) Decision Tree Randomness

I'm starting at Data Science and, to get something going, I just ran the code from Siraj Raval's Intro to Data Science video. He implements a simple Decision Tree Classifier but I couldn't help but notice that, given the same training set, the classifier doesn't always yield the same prediction (nor the same fit apparently); which I happen to find terribly weird, since, from what I've learned, a Decision Tree is supposed to be deterministic.

The only thing I can think of that could be causing the randomness would be that the branches are being chosen at random at some point because 2 options might be identically valued. I would say this could be corrected with a little bit more training data, but even if I add 5 more people, nothing changes. Does anybody have an explanation for what's going on?

Following is the code (in Python) from the video in a for loop to count how many predictions for male and female the Decision Tree has yielded.

from sklearn import tree
from sklearn.svm import SVC

n_male_pred_tree = 0
n_female_pred_tree = 0

n_male_pred_svm = 0
n_female_pred_svm = 0

for i in range (1,1000):
# This loop tests the consistency of the CLF
# The Decision Tree is not very consistent (It's 50-50)
X = [[181,80,44],[177,70,43],[160,60,38],[154,54,37],
[166,65,40],[190,90,47],[175,64,39],[177,70,40],
[159,55,37],[171,75,42],[181,85,43]]

Y = ['male','female','female','female',
'male','male','male','female',
'male','female','male']

tree_clf = tree.DecisionTreeClassifier()
svm_clf = SVC()

tree_clf.fit(X,Y)
svm_clf.fit(X,Y)

tree_prediction = tree_clf.predict([[190,70,43]])
svm_prediction = svm_clf.predict([[190,70,43]])

if tree_prediction == 'male':
n_male_pred_tree += 1
else:
n_female_pred_tree += 1

if svm_prediction == 'male':
n_male_pred_svm += 1
else:
n_female_pred_svm += 1

print(f"MALE pred Tree: {n_male_pred_tree}")
print(f"FEMALE pred for Tree: {n_female_pred_tree}")
print("\n")
print(f"MALE pred for SVM: {n_male_pred_svm}")
print(f"FEMALE pred for SVM: {n_female_pred_svm}")


If you manually set the random_state variable when you create your tree object, you'll find that it does become deterministic.
• Cool! Thanks, I'll have to read the documentation more carefully next time, I guess (although it's difficult with my knowledge right now). I've found this other post which seems to be very in line with my question too. Apparently, the Decision Tree tries to mimic a Random Forest by default, and, as j.a. gartner mentioned, you can change that by fixing the random_state. The classification differs completely depending on the value of random_state (0 or 1). – Philippe Fanaro Aug 9 '18 at 17:27