I'm trying to run a quick univariate filtering on some data, using a t-test of independence, since my target is binary. However, when I run the filter using sklearn
's SelectKBest
, I get the same top features returned doing a manual filter, but in different order. The only information about SelectKBest
I could find is here and the documentation, but both seem like they should work like my manual method. My code is
import numpy as np
from sklearn.feature_selection import SelectKBest
from scipy.stats import ttest_ind
np.random.seed(0)
data = np.random.random((100,50))
target = np.random.randint(2, size = 100).reshape((100,1))
X = data
y = target.ravel()
k = 10
p_values = []
for i in range(data.shape[1]):
t, p = ttest_ind(data[:,i], target)
p_values.append([i,p[0],t[0]])
p_values = sorted(p_values, key = lambda x: x[1])
p_values = p_values[:k]
# Indices of the ranked p-values
idx = [i[0] for i in p_values]
# SelectKBest features
mdl = SelectKBest(ttest_ind, k = k)
X_new = mdl.fit_transform(X, y)
# Manually selected k best features
X_new2=X[:,idx]
# Print first row of sklearn features
print(X_new[0])
array([0.4236548 , 0.96366276, 0.38344152, 0.87001215, 0.63992102,
0.52184832, 0.41466194, 0.06022547, 0.67063787, 0.31542835])
# Print first row of manually selected features
print(X_new2[0])
array([0.67063787, 0.4236548 , 0.31542835, 0.87001215, 0.38344152,
0.63992102, 0.06022547, 0.52184832, 0.41466194, 0.96366276])
It seems SelectKBest
is not ordering the features based solely on their p-values or their t-values. How does SelectKBest
order the features then?