I'm trying to run a quick univariate filtering on some data, using a t-test of independence, since my target is binary. However, when I run the filter using sklearn's SelectKBest, I get the same top features returned doing a manual filter, but in different order. The only information about SelectKBest I could find is here and the documentation, but both seem like they should work like my manual method. My code is

import numpy as np
from sklearn.feature_selection import SelectKBest
from scipy.stats import ttest_ind


data = np.random.random((100,50))
target = np.random.randint(2, size = 100).reshape((100,1))

X = data
y = target.ravel()

k = 10
p_values = []
for i in range(data.shape[1]):

    t, p = ttest_ind(data[:,i], target)

p_values = sorted(p_values, key = lambda x: x[1])
p_values = p_values[:k]

# Indices of the ranked p-values
idx = [i[0] for i in p_values]

# SelectKBest features
mdl = SelectKBest(ttest_ind, k = k)

X_new = mdl.fit_transform(X, y)
# Manually selected k best features

# Print first row of sklearn features
array([0.4236548 , 0.96366276, 0.38344152, 0.87001215, 0.63992102,
   0.52184832, 0.41466194, 0.06022547, 0.67063787, 0.31542835])

# Print first row of manually selected features
array([0.67063787, 0.4236548 , 0.31542835, 0.87001215, 0.38344152,
   0.63992102, 0.06022547, 0.52184832, 0.41466194, 0.96366276])

It seems SelectKBest is not ordering the features based solely on their p-values or their t-values. How does SelectKBest order the features then?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.