# clarification on splitting individual trees in extra trees?

So I am a beginner in machine learning and just started learning about random trees in this article here. When it talks about tuning the hyperparameter K, I'm a bit confused as to how it works. It says:

The parameter K denotes the number of random splits screened at each node to develop Extra-Trees. It may be chosen in the interval [1, ... , n], where n is the number of attributes.

So K would be the number that determines how many attributes to consider for a random split. Then to split, a random attribute from that set will be chosen? But what I'm wondering is that:

If K > 1, in a given set of attributes [1,2,3,4,...,n], is it always a contiguous subset of size K? Or is it K random attributes chosen from those n attributes? And once you choose a random attribute from that subset, it is replaced or left out?

It also says:

For a given problem, the smaller K is, the stronger the randomization of the trees

I'm confused as to why this is.

So K would be the number that determines how many attributes to consider for a random split. Then to split, a random attribute from that set will be chosen?

No, a random split is made for each of the randomly selected features, and these $$K$$ potential splits are scored and the highest scoring one is made.

If K > 1, in a given set of attributes [1,2,3,4,...,n], is it always a contiguous subset of size K? Or is it K random attributes chosen from those n attributes? And once you choose a random attribute from that subset, it is replaced or left out?

We choose $$K$$ random attributes, not necessarily (and probably not) contiguous. I think the original algorithm selects without replacement, and the implementation in sklearn definitely does it without replacement. I could imagine another implementation deciding to do it with replacement though: still the split point for that feature could be different between the two selections. (One more caveat is when a selected feature is constant for the population in the current node: these should be left out, and probably it is implementation-specific whether to select a new feature or just have fewer than $$K$$ in the end.)

It also says:

For a given problem, the smaller K is, the stronger the randomization of the trees

I'm confused as to why this is.

The smaller $$K$$ is, the fewer features you consider for a split, and so the more the algorithm depends on these random choices (and is less likely to latch on to noisy, spurious correlations). Compare to a full decision tree model, in which every split point of every feature is considered.

My 2 cents: I strongly recommend displaying ALL PARAMETERS such as criterion, max_depth, max_leaf_nodes... in order to learn tree algorithms. In my experience, you'll learn a ton such as overfitting.