R vs. Python Decision Tree

Question

From my experiences the R Decision tree returns more accurate results than the python decision tree. Can anymore confirm this assumption and maybe knows the reason?

Answer 1

Decision trees involve a lot of hyperparameters -

• min / max samples in each leaf/leaves

• size

• depth of tree

• criteria for splitting (gini/entropy) etc

Now different packages may have different default settings. Even within R or python if you use multiple packages and compare results, chances are they will be different.

There is nothing which suggests R is "better"

If you want to get the same results, you need to make sure the implicit defaults are similar. For instance, try running the following:

fit <- rpart(y_train ~ ., data = x_train,method="class",
             parms = list(split = "gini"), 
             control = rpart.control(minsplit = 2, minbucket = 1, xval=0, maxdepth = 30))

(predicted5= predict(fit,x_test))
    setosa versicolor virginica
149      0  0.3333333 0.6666667

Here, the parameters minsplit = 2, minbucket = 1, xval=0 and maxdepth = 30 are chosen so as to be identical to the sklearn-options, see here. maxdepth = 30 is the largest value rpart will let you have;

sklearn on the other hand has no bound here. If you want probabilities to be identical, you probably want to play around with the cp parameter as well.

Similarly, with

model = tree.DecisionTreeClassifier(criterion='gini', 
                                    min_samples_split=20, 
                                    min_samples_leaf=round(20.0/3.0), max_depth=30)
model.fit(iris.data, iris.target)

I get

print model.predict([iris.data[49]])
print model.predict([iris.data[99]])
print model.predict([iris.data[100]])
print model.predict([iris.data[149]])
print model.predict([[6.3,2.8,6,1.3]])

[0]
[1]
[2]
[2]
[1]

which looks similar to your initial R output.

All in all, I believe the defaults in R are better suited for the dataset that you are working on, hence the "better" results. But rest assured, they are similar given the parameters are explicit and equal.

Hope this helps!

Answer 2

The main difference is that the R/rpart implementation has post pruning, while scikit learn doesn't. I may lead to a lot of overfitting in case of scikit learn.

Answer 3

It also looks like rpart directly handles categorical variables, but scikit-learn doesn't.

Why do categorical variables need preprocessing in scikit-learn, compared to other tools? Most of scikit-learn assumes data is in NumPy arrays or SciPy sparse matrices of a single numeric dtype. These do not explicitly represent categorical variables at present. Thus, unlike R’s data.frames or pandas.DataFrame, we require explicit conversion of categorical features to numeric values, as discussed in Encoding categorical features. See also Column Transformer with Mixed Types for an example of working with heterogeneous (e.g. categorical and numeric) data.