I'm trying to run a quick univariate filtering on some data, using a t-test of independence, since my target is binary. However, when I run the filter using
SelectKBest, I get the same top features returned doing a manual filter, but in different order. The only information about
SelectKBest I could find is here and the documentation, but both seem like they should work like my manual method. My code is
import numpy as np from sklearn.feature_selection import SelectKBest from scipy.stats import ttest_ind np.random.seed(0) data = np.random.random((100,50)) target = np.random.randint(2, size = 100).reshape((100,1)) X = data y = target.ravel() k = 10 p_values =  for i in range(data.shape): t, p = ttest_ind(data[:,i], target) p_values.append([i,p,t]) p_values = sorted(p_values, key = lambda x: x) p_values = p_values[:k] # Indices of the ranked p-values idx = [i for i in p_values] # SelectKBest features mdl = SelectKBest(ttest_ind, k = k) X_new = mdl.fit_transform(X, y) # Manually selected k best features X_new2=X[:,idx] # Print first row of sklearn features print(X_new) array([0.4236548 , 0.96366276, 0.38344152, 0.87001215, 0.63992102, 0.52184832, 0.41466194, 0.06022547, 0.67063787, 0.31542835]) # Print first row of manually selected features print(X_new2) array([0.67063787, 0.4236548 , 0.31542835, 0.87001215, 0.38344152, 0.63992102, 0.06022547, 0.52184832, 0.41466194, 0.96366276])
SelectKBest is not ordering the features based solely on their p-values or their t-values. How does
SelectKBest order the features then?