What does mtry < 1 in TuneRF mean?

Question

I'm running a random forest model using the randomForest package in R, using the TuneRF function. I have the option to set the 'step factor', which is how much the mtry parameter is changed at each interaction. It allows me to set an step factor of <1 which doesn't make sense to me because I don't see how it can change the number of variables in a model by less than 1? What is going on here?

Answer 1

It's a multiplicative factor, so nonintegers is OK.

But a value <1 shouldn't be used: while theoretically setting it equal to 0.5 should be the same as setting it equal to 2, the details under the hood may break:

      ...
      mtryCur <- if (direction == "left") {
        max(1, ceiling(mtryCur / stepFactor))
      } else {
        min(ncol(x), floor(mtryCur * stepFactor))
      }
      ...

(https://rdrr.io/cran/randomForest/src/R/tuneRF.R)
when stepFactor < 1, the ceiling(mtryCur / stepFactor) may exceed ncol(x), and the ceiling is the wrong direction to round, etc.

See also https://stackoverflow.com/q/27211765/10495893