I would like to ask how to set paramater k in function SelectKBest for feature selection. I have now around 2300 features, so I think that default value 10 is not enough. Is there any approach, how many features choose or it is just on testing to find some compromise between accuracy and number of features?

  • 2
    $\begingroup$ Welcome to this site! How about plotting (model accuracy, k) to find a suitable k? for k = 1, 2, 4, 8, 16, 32, ... $\endgroup$
    – Esmailian
    Commented Mar 28, 2019 at 19:21
  • $\begingroup$ The kind of plot @Esmailian is suggesting is something called an elbow plot. I would also recommend that, using the k spacing @Esmailian suggests. $\endgroup$
    – m13op22
    Commented Apr 12, 2019 at 20:40

2 Answers 2


This is a hyperparameter. And so this is usually approached with some sort of hyperparameters search with cross-validation, like GridSearchCV or RandomizedSearchCV. https://scikit-learn.org/stable/modules/grid_search.html


Since it's a hyperparameter, you should do hyperparameter tuning to determine best number of features K using GridSearch. You can also use SelectPercentile to determine percentile of features to select insted.


from sklearn.model_selection import GridSearchCV
from sklearn.feature_selection import SelectPercentile, SelectKBest
from sklearn.feature_selection import chi2
from sklearn.pipeline import Pipeline

#define your pipeline here
estimator = Pipeline(
    [ ... ,
      ("univ_select", SelectPercentile(chi2)),
      ("estimator", your_estimator())])
grid_params = {
      #define range of percentiles (or k) here
      "univ_select_percentile": [5, 10, 25, 50, 75, 100],
      ... }

# Performing Grid Search
gs = GridSearchCV(estimator,

If you want to use SelectKBest you can generate list of possible number of features using numpy:

import numpy as np
max_features = 2300
min_features = 5
step = 10
grid_params = {
      #define range of k here
      "univ_select_k": np.arange(min_features,max_features,step),
      ... }

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.