(Newbie) Decision Tree Randomness

Question

I'm starting at Data Science and, to get something going, I just ran the code from Siraj Raval's Intro to Data Science video. He implements a simple Decision Tree Classifier but I couldn't help but notice that, given the same training set, the classifier doesn't always yield the same prediction (nor the same fit apparently); which I happen to find terribly weird, since, from what I've learned, a Decision Tree is supposed to be deterministic.

The only thing I can think of that could be causing the randomness would be that the branches are being chosen at random at some point because 2 options might be identically valued. I would say this could be corrected with a little bit more training data, but even if I add 5 more people, nothing changes. Does anybody have an explanation for what's going on?

Following is the code (in Python) from the video in a for loop to count how many predictions for male and female the Decision Tree has yielded.

from sklearn import tree
from sklearn.svm import SVC

n_male_pred_tree = 0
n_female_pred_tree = 0

n_male_pred_svm = 0
n_female_pred_svm = 0

for i in range (1,1000):
    # This loop tests the consistency of the CLF
    # The Decision Tree is not very consistent (It's 50-50)
    X = [[181,80,44],[177,70,43],[160,60,38],[154,54,37],
         [166,65,40],[190,90,47],[175,64,39],[177,70,40],
         [159,55,37],[171,75,42],[181,85,43]]

    Y = ['male','female','female','female',
         'male','male','male','female',
         'male','female','male']

    tree_clf = tree.DecisionTreeClassifier()
    svm_clf = SVC()

    tree_clf.fit(X,Y)
    svm_clf.fit(X,Y)

    tree_prediction = tree_clf.predict([[190,70,43]])
    svm_prediction = svm_clf.predict([[190,70,43]])

    if tree_prediction == 'male':
        n_male_pred_tree += 1
    else:
        n_female_pred_tree += 1

    if svm_prediction == 'male':
        n_male_pred_svm += 1
    else:
        n_female_pred_svm += 1

print(f"MALE pred Tree: {n_male_pred_tree}")
print(f"FEMALE pred for Tree: {n_female_pred_tree}")
print("\n")
print(f"MALE pred for SVM: {n_male_pred_svm}")
print(f"FEMALE pred for SVM: {n_female_pred_svm}")

Answer 1

From sklearn:

The features are always randomly permuted at each split. Therefore, the best found split may vary, even with the same training data and max_features=n_features, if the improvement of the criterion is identical for several splits enumerated during the search of the best split. To obtain a deterministic behaviour during fitting, random_state has to be fixed.

If you manually set the random_state variable when you create your tree object, you'll find that it does become deterministic.

In simpler terms, the data you are feeding it is a little small, and there are several splits that have the same information gain, so the split that is chosen is subject to random factors.