I have a dataset in which I would like to perform a classification model, so I have decided to use Random Forest. The number of features that I have is approximately 200 and I would like to test which set of features gives me a better result. For the sake of experimentation, I do not want to use any feature importance method.
So what I have done is to split my features, without replacement, in groups of 20, 40,60,80 and 100 features. I am using GridSearch for tuning the hyperparameters of my RF. The question that I have is if I want to compare the accuracy of each one of the models (the one with 20 features, with 40, and so on), it would be fair to apply GridSearch with each subset of the features. Or should I only perform Gridsearch once, let's say with the model of 20 features, and then use the same hyperparameters with the set of 40 features, 60 features and so on.
Any help?
Thanks