I have a dataset with 130 features (1000 rows) . I want to select the best features for my classifier. I started with
RFE but Its taking too long, i done this:
number_of_columns = 130 for i in range(1, number_of_columns): rfe = RFE(model, i) fit = rfe.fit(x_train, y_train) acc = fit.score(x_test, y_test
Because this took to long, I changed my approach, and I want to see what you think about it, is it good / correct approach.
First I did
PCA, and I found out that each column participates with around 1-0.4%, except last 9 columns. Last 9 columns participate with less than 0.00001% so I removed them. Now I have 121 features.
pca = PCA()
fit = pca.fit(x)
Then I split my data into train and test (with 121 features).
Then I used
SelectFromModel, and I tested it with 4 different classifiers. Each classifier in
SelectFromModel reduced the number of columns. I chosed the number of column that was determined by classifier that gave me the best accuracy:
model = SelectFromModel(clf, prefit=True) #train_score = clf.score(x_train, y_train) test_score = clf.score(x_test, y_test) column_res = model.transform(x_train).shape
End finally I used 'RFE'. I have used number of columns that i get with 'SelectFromModel'.
rfe = RFE(model, number_of_columns) fit = rfe.fit(x_train, y_train) acc = fit.score(x_test, y_test)
Is this a good approach, or I did something wrong?
Also, If I got the biggest accuracy in
SelectFromModel with one classifier, do I need to use the same classifier in