From my experiences the R Decision tree returns more accurate results than the python decision tree. Can anymore confirm this assumption and maybe knows the reason?
Decision trees involve a lot of hyperparameters -
min / max samples in each leaf/leaves
depth of tree
criteria for splitting (gini/entropy) etc
Now different packages may have different default settings. Even within
python if you use multiple packages and compare results, chances are they will be different.
There is nothing which suggests
R is "better"
If you want to get the same results, you need to make sure the implicit defaults are similar. For instance, try running the following:
fit <- rpart(y_train ~ ., data = x_train,method="class", parms = list(split = "gini"), control = rpart.control(minsplit = 2, minbucket = 1, xval=0, maxdepth = 30)) (predicted5= predict(fit,x_test)) setosa versicolor virginica 149 0 0.3333333 0.6666667
Here, the parameters
minsplit = 2, minbucket = 1, xval=0 and
maxdepth = 30 are chosen so as to be identical to the
sklearn-options, see here.
maxdepth = 30 is the largest value
rpart will let you have;
sklearn on the other hand has no bound here. If you want
probabilities to be identical, you probably want to play around with the
cp parameter as well.
model = tree.DecisionTreeClassifier(criterion='gini', min_samples_split=20, min_samples_leaf=round(20.0/3.0), max_depth=30) model.fit(iris.data, iris.target)
print model.predict([iris.data]) print model.predict([iris.data]) print model.predict([iris.data]) print model.predict([iris.data]) print model.predict([[6.3,2.8,6,1.3]])     
which looks similar to your initial
All in all, I believe the defaults in
R are better suited for the dataset that you are working on, hence the "better" results. But rest assured, they are similar given the parameters are explicit and equal.
Hope this helps!
$\begingroup$ I actually hate this about R. Sometimes the default values do a lot of heavy lifting. And, they’re often reasonable, so you never think to screw with them. $\endgroup$– HEITZMay 11, 2018 at 4:37
The main difference is that the R/rpart implementation has post pruning, while scikit learn doesn't. I may lead to a lot of overfitting in case of scikit learn.
1$\begingroup$ Looks like post-pruning was added in v0.22, it's the ccp_alpha parameter. $\endgroup$ Jul 20, 2022 at 3:14
It also looks like rpart directly handles categorical variables, but scikit-learn doesn't.
Why do categorical variables need preprocessing in scikit-learn, compared to other tools? Most of scikit-learn assumes data is in NumPy arrays or SciPy sparse matrices of a single numeric dtype. These do not explicitly represent categorical variables at present. Thus, unlike R’s data.frames or pandas.DataFrame, we require explicit conversion of categorical features to numeric values, as discussed in Encoding categorical features. See also Column Transformer with Mixed Types for an example of working with heterogeneous (e.g. categorical and numeric) data.