You could use logistic regression with regulation (Lasso, Ridge, Elastic Net) to get an idea of what features are relevant. Regulation means, that features which are „not so important“ are shrunken. In R
glmnet is a good starting point which is also available for Python.
See Section 6.2 in Introduction to statistical learning (ISL) for some background and examples. You may find a PDF of the book online. http://www-bcf.usc.edu/~gareth/ISL/
Ridge is based on the L2 norm, so that model coefficients are shrunken, but cannot be zero. Lasso in contrast uses the L1 norm, so that coefficients can be shrunken to zero.
Another approach to model selection would be forward / backward stepwise selection (see: ISL, Ch. 6.1).
For forward stepwise selection, the process would be:
Let M0 be the NULL model, which containes no predictors. Now for k=0, ..., p-1:
- Consider all p-k models that augment the predictors in M with one additional predictor.
- Choose the best among these p-k models (e.g. based on smalles RSS / best R2).
- Finally, select a single best model from the M models using cross-validated prediction error, AIC, BIC, or adjusted R2.
Backward stepwise selection works in the same way, but you start with the full model and successively leave out predictors/features.
For Python, sklearn comes with a greedy stepwise (backward) selection module: https://scikit-learn.org/stable/modules/generated/sklearn.feature_selection.RFE.html
Also note, that many methods/packages come with a module that allows to assess feature importance, e.g. in catboost (https://catboost.ai/docs/concepts/fstr.html) or lightgbm (https://lightgbm.readthedocs.io/en/latest/Python-API.html). However, a disadvantage here is, that you need to implement a proper model (with some hyperparameter tuning) first, which is time consuming.