I am stuck on a theoretical roadblock in learning about machine learning, because I have not seen this explicitly addressed anywhere. In my studies, it seems as if Cross-validation (or some variant thereof, like LOOCV, or potentially another, but similar, validation scheme like bootstrapping) is the be-all-end-all of model selection. Choosing models and their parameters via exhaustive CV to maximize fit but also balance overfitting seems the optimal way to create models, and computational power is only getting cheaper. So what is there left to do for the human analyst?
I apologize in advance for this amateurish question, but could anyone fill in this gap for me, and potentially suggest some sources on model selection?