Why can't we sample without replacement for each tree in a random forest if the subsample size is large enough?

Usually if we have $$n$$ observations, for each tree with form a bootstrapped subsample of size $$n$$ with replacement. On googling it one common explanation I've seen is that with replacement sampling is necessary for independence of individual trees.

But why can't we just resample as follows: for tree 1, randomly sample $$m$$ observations without replacement out of the $$n$$, where $$m$$ is still large enough (of course, provided that $$n$$ is large enough in the first place). Then replenish all observations and repeat the resampling for tree 2, and so on.

Even in this case, I'd imagine that the individual subsamples would be independent. So is there an additional reason for resampling with replacement in bagging?

For example, imagine a class-imbalanced binary problem, once the minority class is already sampled (large possibility that this can happen given $$n$$ and $$m$$) then, without replacement, the rest trees will only sample from the majority class which will produce skewed trees.