Weka: Implementation of Random Forest

Question

I am wondering how random forests are exactly implemented in Weka. This paper is very specific about RFs in Weka, but the description of its learning process in chapter 2 seems strange to me. They say:

1. Bootstrap samples $B_i$ for every tree $t_i$
2. A random subset of features is selected for each $t_i$
3. Information gain is used to grow unpruned trees $t_i$

My questions:

• Shouldn't step 2 be repeated on all levels of the decision tree? Otherwise each tree will never see some of the features
• Whats the default when setting numFeatures=0. I think this is the number of features that is available for each split. Is it the square root of the number of all features?
• Is really information gain used for determining the best split attribute?

I am using Weka 3.8.3 - not sure if this matters.

Thanks for all hints :)

Answer 1

Your linked paper appears to be wrong about feature subsetting. I couldn't find it in the documentation for randomForest, but the source for randomForest uses randomTree for the base models, and in that documentation it says

a tree that considers K randomly chosen attributes at each node.

So the selection seems to happen at each split.
(Note that xgboost has feature subsetting at each of the tree, the level (depth), and the node. I don't see any obvious reason that one or more of these options should always be preferable...)

For default number of features, sqrt(m) is the most common, but it looks like Weka uses lg(m). See option -K at
https://weka.sourceforge.io/doc.dev/weka/classifiers/trees/RandomForest.html

Yes, Weka uses the Quinlan family of decision trees, which split using information gain (as opposed to CART, which uses gini).