I have a dataset with 261 predictors scraped from a larger set of survey questions. 224 have values which are in a range of scale (some 1-10, some 1-4, some simply binary, all using 0 where no value is given), and the rest are unordered categories.
I'm trying to perform classification using these predictors and identify the top n predictors. Am thinking of the following approach:
- convert the 224 ordered predictors into numeric, centered, and scaled.
- Run separate modeling (I use caret from R): one for using the numeric predictors, another using the remaining 37 categorical predictors (both cross-validated within each modeling exercise).
- Choose the respective best-fitting models modelN and modelC for the numeric and categorical predictors.
- Choose top n (say 10) predictors from model N and model C.
- Combine them in an ensemble model that can handle both numeric and categorical data (say, random forest).
- Choose top n predictors in the ensemble model.
I am going through this a roundabout way rather than directly fitting all predictors into an ensemble model to try and reduce the complexity of the problem first (and because in R, I'm having a problem with too many levels from the predictors).
Would this be a valid approach to identifying the n most salient predictors? Any possible issues to mitigate?