So I am a beginner in machine learning and just started learning about random trees in this article here. When it talks about tuning the hyperparameter K, I'm a bit confused as to how it works. It says:
The parameter K denotes the number of random splits screened at each node to develop Extra-Trees. It may be chosen in the interval [1, ... , n], where n is the number of attributes.
So K would be the number that determines how many attributes to consider for a random split. Then to split, a random attribute from that set will be chosen? But what I'm wondering is that:
If K > 1, in a given set of attributes [1,2,3,4,...,n], is it always a contiguous subset of size K? Or is it K random attributes chosen from those n attributes? And once you choose a random attribute from that subset, it is replaced or left out?
It also says:
For a given problem, the smaller K is, the stronger the randomization of the trees
I'm confused as to why this is.