Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
At first, I did a GridsearchCV and the best parameter I found was 100, i.e., a random forest with just 100 trees. My trainset has 80,000 rows and 669 columns. My test set has 20,000 rows and 669 columns.
How is it possible that such small number of trees is enough?
By other posts and this one seems what you don't have a clear intuition of the n_estimators of the random forest.
I am going to assume that you are referring to the n_estimators (from this other question). n_estimators is the number of trees that your 'forest' has. Not the depth of your tree. That is another parameter.
If you are referring to max_depth = 100, 100 splits it can be a lot. Feel free to plot one tree and see what it is doing. see this link
The number of rows and columns is not necessarily important, what is important is the complexity of your problem.
My 2 cents: I'm fan of defying the max_leaf_nodes (in this example 5) and then visualizing it. I suggest starting at 3 and then increasing it slightly (the same applies for your Random Forest). In general, at around 5 I see overfitting. With your large dataset, you might need a bit more (i.e. max_leaf_nodes = 10?).
Why? Or the answer to your question... Those tree-based algorithms are able to catch highly non-linear problems pretty quickly, but then start overfitting.
