I have a classification problem where my inputs are all numerical and continuous, my outputs are categorical labels [1,0,-1]
. My own domain knowledge and understanding of ML workflow tells me some features are redundant and don't provide much predictive insight to my model.
I am using Sklearns SelectKBest()
package. However there are numerous statistical tests to conduct such methods. So my question is, given my inputs and outputs, which statistical method is best for my situation?
Furthermore, I wish to implement this step in Sklearns pipeline()
package with GridSearch. My pipeline is quite rudimentary at this point but I still don't know which step this should be conducted. Would appreciate your answer guys.
My pipeline code is as follows:
pipe = Pipeline([('sc', preprocessing.MinMaxScaler()),
('SVM', svm.SVC(decision_function_shape = 'ovr', kernel = 'poly'))])
candidate_parameters = [{'SVM__C': [0.01, 0.1, 1, 10], 'SVM__gamma': [0.01, 0.1, 1, 10]}]
print('Fitting model...')
clf = GridSearchCV(estimator = pipe, param_grid = candidate_parameters, cv = 5, n_jobs = -1)
clf.fit(X_train, y_train )