I have a dataset with 130 features (1000 rows) . I want to select the best features for my classifier. I started with RFE
but Its taking too long, i done this:
number_of_columns = 130
for i in range(1, number_of_columns):
rfe = RFE(model, i)
fit = rfe.fit(x_train, y_train)
acc = fit.score(x_test, y_test
Because this took to long, I changed my approach, and I want to see what you think about it, is it good / correct approach.
First I did PCA
, and I found out that each column participates with around 1-0.4%, except last 9 columns. Last 9 columns participate with less than 0.00001% so I removed them. Now I have 121 features.
pca = PCA()
fit = pca.fit(x)
Then I split my data into train and test (with 121 features).
Then I used SelectFromModel
, and I tested it with 4 different classifiers. Each classifier in SelectFromModel
reduced the number of columns. I chosed the number of column that was determined by classifier that gave me the best accuracy:
model = SelectFromModel(clf, prefit=True)
#train_score = clf.score(x_train, y_train)
test_score = clf.score(x_test, y_test)
column_res = model.transform(x_train).shape
End finally I used 'RFE'. I have used number of columns that i get with 'SelectFromModel'.
rfe = RFE(model, number_of_columns)
fit = rfe.fit(x_train, y_train)
acc = fit.score(x_test, y_test)
Is this a good approach, or I did something wrong?
Also, If I got the biggest accuracy in SelectFromModel
with one classifier, do I need to use the same classifier in RFE
?
ranking_
attribute. Or, useRFECV
. $\endgroup$