From my experiences the R Decision tree returns more accurate results than the python decision tree. Can anymore confirm this assumption and maybe knows the reason?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
Decision trees involve a lot of hyperparameters -
min / max samples in each leaf/leaves
size
depth of tree
criteria for splitting (gini/entropy) etc
Now different packages may have different default settings. Even within R or python if you use multiple packages and compare results, chances are they will be different.
There is nothing which suggests R is "better"
If you want to get the same results, you need to make sure the implicit defaults are similar. For instance, try running the following:
fit <- rpart(y_train ~ ., data = x_train,method="class",
parms = list(split = "gini"),
control = rpart.control(minsplit = 2, minbucket = 1, xval=0, maxdepth = 30))
(predicted5= predict(fit,x_test))
setosa versicolor virginica
149 0 0.3333333 0.6666667
Here, the parameters minsplit = 2, minbucket = 1, xval=0 and maxdepth = 30 are chosen so as to be identical to the sklearn-options, see here. maxdepth = 30 is the largest value rpart will let you have;
sklearn on the other hand has no bound here. If you want probabilities to be identical, you probably want to play around with the cp parameter as well.
Similarly, with
model = tree.DecisionTreeClassifier(criterion='gini',
min_samples_split=20,
min_samples_leaf=round(20.0/3.0), max_depth=30)
model.fit(iris.data, iris.target)
I get
print model.predict([iris.data[49]])
print model.predict([iris.data[99]])
print model.predict([iris.data[100]])
print model.predict([iris.data[149]])
print model.predict([[6.3,2.8,6,1.3]])
[0]
[1]
[2]
[2]
[1]
which looks similar to your initial R output.
All in all, I believe the defaults in R are better suited for the dataset that you are working on, hence the "better" results. But rest assured, they are similar given the parameters are explicit and equal.
Hope this helps!
answered May 9, 2018 at 8:51
-
$\begingroup$ I actually hate this about R. Sometimes the default values do a lot of heavy lifting. And, they’re often reasonable, so you never think to screw with them. $\endgroup$– HEITZMay 11, 2018 at 4:37
-
$\begingroup$ It looks like it is not easy to replicate
cpin python: "The lambda is determined through cross-validation and not reported in R." source: learnbymarketing.com/tutorials/rpart-decision-trees-in-r/…. $\endgroup$ May 5 at 16:48
$\begingroup$
$\endgroup$
1
The main difference is that the R/rpart implementation has post pruning, while scikit learn doesn't. I may lead to a lot of overfitting in case of scikit learn.
-
2$\begingroup$ Looks like post-pruning was added in v0.22, it's the ccp_alpha parameter. $\endgroup$ Jul 20, 2022 at 3:14
$\begingroup$
$\endgroup$
It also looks like rpart directly handles categorical variables, but scikit-learn doesn't.
Why do categorical variables need preprocessing in scikit-learn, compared to other tools? Most of scikit-learn assumes data is in NumPy arrays or SciPy sparse matrices of a single numeric dtype. These do not explicitly represent categorical variables at present. Thus, unlike R’s data.frames or pandas.DataFrame, we require explicit conversion of categorical features to numeric values, as discussed in Encoding categorical features. See also Column Transformer with Mixed Types for an example of working with heterogeneous (e.g. categorical and numeric) data.
