0
$\begingroup$

I am training a Neural Network for Multi-Class classification. After succesfully training it and validating the model through cross validation, I would like to use this network inside a voting Classifier. In order to perform cross validation on my trained network I convert it to a Keras Classifier and then calculate its validation score. However when I parse the same exact "Keras Classifier" in the Voting Classifier method, I get the following error:

ValueError: The estimator KerasClassifier should be a classifier

The code can be seen below:

import random 
random.seed(42)

from keras.layers import Dense
from keras.models import Sequential
from keras.wrappers.scikit_learn import KerasClassifier
import tensorflow as tf
from sklearn.ensemble import VotingClassifier

def NeuralNetwork():
    model = Sequential()
    # define first hidden layer and visible layer
    model.add(Dense(600, input_dim=k, activation='relu'))
    model.add(Dense(20, activation='relu'))
    # define output layer
    model.add(Dense(3, activation='softmax'))
    # define loss and optimizer
    model.compile(loss='sparse_categorical_crossentropy', optimizer='adam', metrics=["accuracy"])
    return model

NN = KerasClassifier(build_fn=NeuralNetwork , epochs=100, batch_size=100, verbose=0)
cross_val_score(NN, X_new_train,y_train_no_id, cv=3)


votingC = VotingClassifier(estimators=[ ('LR1', LR1),('LR2', LR2), ('XGB',XGB),('NN',NN)], voting='hard', n_jobs=4)   
#LR1 and LR2 are some other Logistic Regression estimators defined in another section
votingC = votingC.fit(X_new_train, y_train_no_id)
votingC.predict(X_new_test)
$\endgroup$
0
$\begingroup$

There are a couple of Github issues on this - see here and here. In short, while scikit-learn native models include an _estimator_type attribute:

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression()
clf._estimator_type
# 'classifier'

this is not the case with a KerasClassifier; using your own NN gives

NN = KerasClassifier(build_fn=NeuralNetwork , epochs=100, batch_size=100, verbose=0)
NN._estimator_type

---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)

<ipython-input-4-a2fd193c154f> in <module>()
----> 1 NN._estimator_type

AttributeError: 'KerasClassifier' object has no attribute '_estimator_type'

As reported in a comment here, setting the attribute manually seems to work:

NN._estimator_type = "classifier"

so add this line immediately after your NN definition.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.