8
$\begingroup$

From my experiences the R Decision tree returns more accurate results than the python decision tree. Can anymore confirm this assumption and maybe knows the reason?

$\endgroup$

3 Answers 3

7
$\begingroup$

Decision trees involve a lot of hyperparameters -

  • min / max samples in each leaf/leaves

  • size

  • depth of tree

  • criteria for splitting (gini/entropy) etc

Now different packages may have different default settings. Even within R or python if you use multiple packages and compare results, chances are they will be different.

There is nothing which suggests R is "better"

If you want to get the same results, you need to make sure the implicit defaults are similar. For instance, try running the following:

fit <- rpart(y_train ~ ., data = x_train,method="class",
             parms = list(split = "gini"), 
             control = rpart.control(minsplit = 2, minbucket = 1, xval=0, maxdepth = 30))

(predicted5= predict(fit,x_test))
    setosa versicolor virginica
149      0  0.3333333 0.6666667

Here, the parameters minsplit = 2, minbucket = 1, xval=0 and maxdepth = 30 are chosen so as to be identical to the sklearn-options, see here. maxdepth = 30 is the largest value rpart will let you have;

sklearn on the other hand has no bound here. If you want probabilities to be identical, you probably want to play around with the cp parameter as well.

Similarly, with

model = tree.DecisionTreeClassifier(criterion='gini', 
                                    min_samples_split=20, 
                                    min_samples_leaf=round(20.0/3.0), max_depth=30)
model.fit(iris.data, iris.target)

I get

print model.predict([iris.data[49]])
print model.predict([iris.data[99]])
print model.predict([iris.data[100]])
print model.predict([iris.data[149]])
print model.predict([[6.3,2.8,6,1.3]])

[0]
[1]
[2]
[2]
[1]

which looks similar to your initial R output.

All in all, I believe the defaults in R are better suited for the dataset that you are working on, hence the "better" results. But rest assured, they are similar given the parameters are explicit and equal.

Hope this helps!

$\endgroup$
2
  • $\begingroup$ I actually hate this about R. Sometimes the default values do a lot of heavy lifting. And, they’re often reasonable, so you never think to screw with them. $\endgroup$
    – HEITZ
    May 11, 2018 at 4:37
  • $\begingroup$ It looks like it is not easy to replicate cp in python: "The lambda is determined through cross-validation and not reported in R." source: learnbymarketing.com/tutorials/rpart-decision-trees-in-r/…. $\endgroup$ May 5 at 16:48
5
$\begingroup$

The main difference is that the R/rpart implementation has post pruning, while scikit learn doesn't. I may lead to a lot of overfitting in case of scikit learn.

$\endgroup$
1
  • 2
    $\begingroup$ Looks like post-pruning was added in v0.22, it's the ccp_alpha parameter. $\endgroup$ Jul 20, 2022 at 3:14
5
$\begingroup$

It also looks like rpart directly handles categorical variables, but scikit-learn doesn't.

Why do categorical variables need preprocessing in scikit-learn, compared to other tools? Most of scikit-learn assumes data is in NumPy arrays or SciPy sparse matrices of a single numeric dtype. These do not explicitly represent categorical variables at present. Thus, unlike R’s data.frames or pandas.DataFrame, we require explicit conversion of categorical features to numeric values, as discussed in Encoding categorical features. See also Column Transformer with Mixed Types for an example of working with heterogeneous (e.g. categorical and numeric) data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.