1
$\begingroup$

At first, I did a GridsearchCV and the best parameter is a random forest with just 100 trees. My trainset has 80.000 rows and 669 columns. My test set has 20.000 rows and 669 columns.

How is it possible that so low trees are enough?

$\endgroup$
2
$\begingroup$

By other posts and this one seems what you don't have a clear intuition of the n_estimators of the random forest.

I am going to assume that you are referring to the n_estimators (from this other question). n_estimators is the number of trees that your 'forest' has. Not the depth of your tree. That is another parameter.

If you are referring to max_depth = 100, 100 splits it can be a lot. Feel free to plot one tree and see what it is doing. see this link

The number of rows and columns is not necessarily important, what is important is the complexity of your problem.

| improve this answer | |
$\endgroup$
2
$\begingroup$

My 2 cents: I'm fan of defying the max_leaf_nodes (in this example 5) and then visualizing it. I suggest starting at 3 and then increasing it slightly (the same applies for your Random Forest). In general, at around 5 I see overfitting. With your large dataset, you might need a bit more (i.e. max_leaf_nodes = 10?).

Why? Or the answer to your question... Those tree-based algorithms are able to catch highly non-linear problems pretty quickly, but then start overfitting.

enter image description here

enter image description here

| improve this answer | |
$\endgroup$
  • $\begingroup$ I am also becoming a fan of visualizing random forest trees. I even sometimes try to make new features of the results of the visualization $\endgroup$ – Carlos Mougan Jan 25 at 15:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.