0
$\begingroup$

I'm trying to run a quick univariate filtering on some data, using a t-test of independence, since my target is binary. However, when I run the filter using sklearn's SelectKBest, I get the same top features returned doing a manual filter, but in different order. The only information about SelectKBest I could find is here and the documentation, but both seem like they should work like my manual method. My code is

import numpy as np
from sklearn.feature_selection import SelectKBest
from scipy.stats import ttest_ind

np.random.seed(0)

data = np.random.random((100,50))
target = np.random.randint(2, size = 100).reshape((100,1))

X = data
y = target.ravel()

k = 10
p_values = []
for i in range(data.shape[1]):

    t, p = ttest_ind(data[:,i], target)
    p_values.append([i,p[0],t[0]])

p_values = sorted(p_values, key = lambda x: x[1])
p_values = p_values[:k]

# Indices of the ranked p-values
idx = [i[0] for i in p_values]

# SelectKBest features
mdl = SelectKBest(ttest_ind, k = k)

X_new = mdl.fit_transform(X, y)
# Manually selected k best features
X_new2=X[:,idx]


# Print first row of sklearn features
print(X_new[0])
array([0.4236548 , 0.96366276, 0.38344152, 0.87001215, 0.63992102,
   0.52184832, 0.41466194, 0.06022547, 0.67063787, 0.31542835])

# Print first row of manually selected features
print(X_new2[0])
array([0.67063787, 0.4236548 , 0.31542835, 0.87001215, 0.38344152,
   0.63992102, 0.06022547, 0.52184832, 0.41466194, 0.96366276])

It seems SelectKBest is not ordering the features based solely on their p-values or their t-values. How does SelectKBest order the features then?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.