Here are my two cents.
1) While there are some things that you may want to do differently (see part 2), what you observe is normal. If you had a trivial problem with a simple landscape, all of the methods would converge on exactly one subset of features. Feature selection in the real world, however, is rarely a trivial problem (as long as you are looking for "true" optimality and not a "good enough" solution). You are facing a search space of size $2^{60}$, which is impossible to assess in its entirety, so navigation methods will vary. RFECV
is a top-down approach that attempts to shed as many weak features as possible based on the CV scores, SelectFromModel
relies on estimator's internal objectives to do the selection, while SelectKBest
in its default form is a simple filter that computes ANOVA F-values without considering feature interactions. Since the criteria are so different, you are bound to end up with different feature subsets -- each method operates on a landscape that is "distorted" by its objective function. The fact that the subsets you have found intersect suggests that the common features possess some sort of optimality (when all used approaches are considered). While you are at it, consider sequential feature selection methods, especially the floating approaches (both bottom-up and top-down), as they can be a great deal more effective than plain RFE
/ KBest
.
2) You are using SMOTE
to balance the training data. From your problem statement it seems that you are introducing synthetic data points before feature selection. If this is the case, then please consider the implications: for RFECV
(if you do something like rfecv.fit(X_train, y_train)
), the validation sample is not drawn from the original distribution, but from the one already affected by synthetic data, so the CV scores that drive the selection are not based on the true qualities of the features. This discrepancy may be too small to actually affect anything, but the fact remains. In your case, the class balance is 70/30, which is not that bad. Do you actually observe any increase in performance after balancing? As far as metrics go, the area under Precision-Recall curve is stable irrespective of the balance.
3) Decision trees (extensively used in many flavors, including XGBoost and CatBoost that you mentioned) have feature selection embedded into the process of learning. This means you can just plug in full featured data into the learner and have it figure out the most important features completely on its own. It's very effective and usually sufficient. However, in order to have variety of opinions within the ensemble, forest methods (most of them, anyway) will give all features an equal chance of being used, so it's very likely that you will end up with a situation where all features have some importance -- you will still have to decide which ones to keep. Weak features with noise that correlates with your labels by pure chance can affect your model's ability to generalize and they can even trick you into keeping them as useful. Also, when your sample size is limited (and it always is), increasing the number of features does not help the confidence of your assessments, either. Without enough observations, how can you be sure that the label is what it is because of this one feature and not ten others? So delegating it all to the forest without any preliminary or followup steps is not necessarily the best course of action.
The answer to the general question of the post: How to Maximize recall for Minority class?
This is quite straightforward. Define recall on minority class as your objective, i.e. let it be the output of the scoring function that you supply to your feature selection algorithm. For sklearn's feature selection methods it's the score_func parameter, for mlextend's SFS it's scoring. The resulting feature subsets are likely to contain something of interest.
Only after performing a thorough search for optimal features, selecting the best kind of model for your problem and having tuned its hyperparameters, if you still for whatever reason want to push your precision/recall further, you may want to look into stacking -- this will squeeze out the last bits of performance juice at the expense of model complexity. But the chances are that it's really not worth it.