Does anyone know how the python sklearn random forest implementation handles continuous variables in the fitting process? I'm curious to know if it does any sort of binning (and if so, how it does the binning), or if a continuous variable is just treated as a categorical variable? I'm hoping it's not the latter...thanks! Also, I'd be open to using some R implementation if anyone knows about that.
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
To understand how a random forest treats continuous data it is imperative to understand how a random forest works. At the base of the random forest algorithm lays a tree construction. The default in sklearn is to split a tree based on the Gini coefficient (see sklearn documentation). This type of tree algorithm is referred to as CART trees. You can change the criterion to entropy to select ID3 and C4.5 trees. Without going to deep into the maths, the tree algorithm will seek to split the tree based on a cutoff that leads to the lowest Gini coefficient.
The random forest algorithm will build a large number of deep trees on your data and average over all the trained trees to give you the final prediction.
Depending on your requirements in terms of data size and necessity for parallelization I can highly recommend H2O. It is an open source machine learning software suite with APIs in Python and R. Their random forest implementation is very fast and leads to models with a higher AUC (see this page for a good comparison between different ML libraries).
-
$\begingroup$ Yeah, well i thought I understood decision trees lol. Does splitting based on Gini coefficient create a binary split? I thought decision trees didn't have to be binary? $\endgroup$– Ben FOct 19, 2016 at 15:15
-
$\begingroup$ Ok I read more about Gini impurity, and does only do binary splits. Thanks! $\endgroup$– Ben FOct 19, 2016 at 15:57