I'm using the GridSearchCV ()
class from scikit to perform hyperparameter optimization in a sequential neural network. I've built a pipeline to also find the best number of features by putting a feature selector inside the pipeline. The problem is how to define the input_shape
, since this depends on the k
parameter from the feature selector. Is it possible to set the value of classifier__input_shape
to be the same value (at all times) of feature_selector__feature__selector_k
?
I've provided the correspondent piece of code below.
def create_model (learn_rate = 0.01, dropout_rate = 0.0, weight_constraint = 0, input_shape):
model = Sequential ()
model.add (Dense (units = 64, activation = 'relu',
input_shape = (input_shape, )))
model.add (Dropout (dropout_rate))
model.add (Dense (32, activation = 'relu'))
model.add (Dense (1, activation = 'sigmoid'))
model.compile (loss = 'binary_crossentropy',
optimizer = Adam (lr = learn_rate),
metrics = ['accuracy'])#, metrics.CategoricalAccuracy ()])
return model
standard_scaler_features = remaining_features
my_scaler = StandardScaler ()
steps = list ()
steps.append (('scaler', my_scaler))
standard_scaler_transformer = Pipeline (steps)
my_feature_selector = SelectKBest ()
steps = list ()
steps.append (('feature_selector', my_feature_selector))
feature_selector_transformer = Pipeline (steps)
clf = KerasClassifier (build_fn = create_model, verbose = 2)
clf = Pipeline (steps = [('scaler', my_scaler),
('feature_selector', feature_selector_transformer),
('classifier', clf)],
verbose = True)
param_grid = {'feature_selector__feature_selector__score_func' : [f_classif],
'feature_selector__feature_selector__k' : [7, 9, 15],
'classifier__input_shape' : [7, 9, 15],
'classifier__epochs' : [2, 3, 4]}
cv = RepeatedStratifiedKFold (n_splits = 5, n_repeats = 1, random_state = STATE)
grid = GridSearchCV (estimator = clf, param_grid = param_grid, scoring = 'f1',
verbose = 1, n_jobs = 1, cv = cv)
grid_result = grid.fit (X_train_df, y_train_df)
And the error:
ValueError: Input 0 of layer sequential_9 is incompatible with the layer: expected axis -1 of input shape to have value 9 but received input with shape [None, 7]