One critique I have of statistical learning (SL) pedagogy is the absence of computation performance considerations in the evaluation of different modeling techniques. With its emphases on bootstrapping and cross-validation to tune/test models, SL is quite compute-intensive. Add to that the re-sampling that's embedded in techniques like bagging and boosting, and you have the specter of computation hell for supervised learning of large data sets. In fact, R's memory constraints impose pretty severe limits on the size of models that can be fit by top-performing methods like random forests. Though SL does a good job calibrating model performance against small data sets, it'd sure be nice to understand performance versus computational cost for larger data.
What are R's memory constraints, and do they impose severe limits on the size of models that can be fit by top-performing methods like random forests?