I'm starting at Data Science and, to get something going, I just ran the code from Siraj Raval's Intro to Data Science video. He implements a simple Decision Tree Classifier but I couldn't help but notice that, given the same training set, the classifier doesn't always yield the same prediction (nor the same fit apparently); which I happen to find terribly weird, since, from what I've learned, a Decision Tree is supposed to be deterministic.
The only thing I can think of that could be causing the randomness would be that the branches are being chosen at random at some point because 2 options might be identically valued. I would say this could be corrected with a little bit more training data, but even if I add 5 more people, nothing changes. Does anybody have an explanation for what's going on?
Following is the code (in Python) from the video in a for
loop to count how many predictions for male and female the Decision Tree has yielded.
from sklearn import tree
from sklearn.svm import SVC
n_male_pred_tree = 0
n_female_pred_tree = 0
n_male_pred_svm = 0
n_female_pred_svm = 0
for i in range (1,1000):
# This loop tests the consistency of the CLF
# The Decision Tree is not very consistent (It's 50-50)
X = [[181,80,44],[177,70,43],[160,60,38],[154,54,37],
[166,65,40],[190,90,47],[175,64,39],[177,70,40],
[159,55,37],[171,75,42],[181,85,43]]
Y = ['male','female','female','female',
'male','male','male','female',
'male','female','male']
tree_clf = tree.DecisionTreeClassifier()
svm_clf = SVC()
tree_clf.fit(X,Y)
svm_clf.fit(X,Y)
tree_prediction = tree_clf.predict([[190,70,43]])
svm_prediction = svm_clf.predict([[190,70,43]])
if tree_prediction == 'male':
n_male_pred_tree += 1
else:
n_female_pred_tree += 1
if svm_prediction == 'male':
n_male_pred_svm += 1
else:
n_female_pred_svm += 1
print(f"MALE pred Tree: {n_male_pred_tree}")
print(f"FEMALE pred for Tree: {n_female_pred_tree}")
print("\n")
print(f"MALE pred for SVM: {n_male_pred_svm}")
print(f"FEMALE pred for SVM: {n_female_pred_svm}")