2
$\begingroup$

I am adapting sklearn-extension ELMClassifier to be accepted as base_estimator to both VotingClassifier and AdaboostClassifier. My code works fine when i use my ELM directly with AdaboostClassifier, but since i'm going to create an Adaboost of different classifiers, i need it to be instantiated inside of VotingClassifier and then pass the VotingClassifier to the Adaboost as a base estimator.

When i run adaboostCLF.fit(base_estimator=votingCLF, algorithm="SAMME"), i get the following error log:

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-78-25ca9de0ea07> in <module>
          1 ada2 = AdaBoostClassifier(base_estimator=votingCLF, algorithm='SAMME')
    ----> 2 ada2.fit(X,y)

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_weight_boosting.py in fit(self, X, y, sample_weight)
        436 
        437         # Fit
    --> 438         return super().fit(X, y, sample_weight)
        439 
        440     def _validate_estimator(self):

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_weight_boosting.py in fit(self, X, y, sample_weight)
        140                 X, y,
        141                 sample_weight,
    --> 142                 random_state)
        143 
        144             # Early termination

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_weight_boosting.py in _boost(self, iboost, X, y, sample_weight, random_state)
        499         else:  # elif self.algorithm == "SAMME":
        500             return self._boost_discrete(iboost, X, y, sample_weight,
    --> 501                                         random_state)
        502 
        503     def _boost_real(self, iboost, X, y, sample_weight, random_state):

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_weight_boosting.py in _boost_discrete(self, iboost, X, y, sample_weight, random_state)
        563         estimator = self._make_estimator(random_state=random_state)
        564 
    --> 565         estimator.fit(X, y, sample_weight=sample_weight)
        566 
        567         y_predict = estimator.predict(X)

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_voting.py in fit(self, X, y, sample_weight)
        220         transformed_y = self.le_.transform(y)
        221 
    --> 222         return super().fit(X, transformed_y, sample_weight)
        223 
        224     def predict(self, X):

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_voting.py in fit(self, X, y, sample_weight)
         55     def fit(self, X, y, sample_weight=None):
         56         """Get common fit operations."""
    ---> 57         names, clfs = self._validate_estimators()
         58 
         59         if (self.weights is not None and

    ~\Anaconda3\lib\site-packages\sklearn\ensemble\_base.py in _validate_estimators(self)
        247                 raise ValueError(
        248                     "The estimator {} should be a {}.".format(
    --> 249                         est.__class__.__name__, is_estimator_type.__name__[3:]
        250                     )
        251                 )

    ValueError: The estimator customELMClassifer should be a classifier.

And here is the code for the ELM i'm using:

    class customELMClassifer(ELMClassifier):
        def resample_with_replacement(self, X_train, y_train, sample_weight):

            # normalize sample_weights if not already
            sample_weight = sample_weight / sample_weight.sum(dtype=np.float64)

            X_train_resampled = np.zeros((len(X_train), len(X_train[0])), dtype=np.float32)
            y_train_resampled = np.zeros((len(y_train)), dtype=np.int)
            for i in range(len(X_train)):
                # draw a number from 0 to len(X_train)-1
                draw = np.random.choice(np.arange(len(X_train)), p=sample_weight)

                # place the X and y at the drawn number into the resampled X and y
                X_train_resampled[i] = X_train[draw]
                y_train_resampled[i] = y_train[draw]

            return X_train_resampled, y_train_resampled


        def fit(self, X, y, sample_weight=None, random_state=0):
            if sample_weight is not None:
                X, y = self.resample_with_replacement(X, y, sample_weight)

            return super().fit(X, y)

I'd appreciate some feedback or solution since i have no experience creating scikit-learn estimators from scratch.

Thanks in advance.

$\endgroup$
2
  • $\begingroup$ I am not sure AdaBoostClassifier, or any other scikit-learn kernel, accepts any classifier as base_estimator, and it might not accept ELMClassifier, at least for the moment. You should open an issue on Github and ask directly the scikit-learn community about that. $\endgroup$ Mar 20 '20 at 13:36
  • 1
    $\begingroup$ It does, the default is DecisionTreeClassifier. If the estimator implements sample_weight, then it can be used in Adaboost. $\endgroup$ Mar 21 '20 at 15:37
1
$\begingroup$

that's probably due to the fact your class is not really 100% compatible to the scikit-learn estimator interface. You can easily verify this with the check_estimator method in sklearn.utils.estimator_checks. This should ensure you it is a proper classifier which can be passed then to AdaBoost.

I'd also suggest to inherit from BaseEstimator in addition to ELMClassifier.

For further details, see the instructions reported here in order to create a custom (compatible to scikit-learn interface) estimator.

$\endgroup$
0
$\begingroup$

From the traceback, you can find the problem stems from here:

is_estimator_type = (is_classifier if is_classifier(self)
                     else is_regressor)

for est in estimators:
    if est not in (None, 'drop') and not is_estimator_type(est):
        raise ValueError(
            "The estimator {} should be a {}.".format(
            est.__class__.__name__, is_estimator_type.__name__[3:]
        )
    )

where the definition for is_classifier is just

getattr(estimator, "_estimator_type", None) == "classifier"

Normally, inheriting ClassifierMixin is good practice and will provide the attribute _estimator_type = "classifier". In this case there may be complications since ELMClassifier inherits from ELMRegressor; probably the easiest way to work around it is to add estimator_type = "classifier" to your customELMClassifier.

$\endgroup$
0
$\begingroup$

I had this problem and it came from not returning the clf from my classifier functions so when I was doing:

def random_for():
    from sklearn.ensemble import RandomForestClassifier
    clf = RandomForestClassifier(max_depth = 5)
    clf.fit(Input, Labels)
    y_pred = clf.predict(Test_input)
    predictions = [round(value) for value in y_pred]
    accuracy = accuracy_score(Test_Labels, predictions)
    print("Random Forest accuracy: %.2f%%" % (accuracy*100.0))

rclf = random_for()

I got the same error as the OP. Whereas;

def random_for():
    from sklearn.ensemble import RandomForestClassifier
    clf = RandomForestClassifier(max_depth = 5)
    clf.fit(Input, Labels)
    y_pred = clf.predict(Test_input)
    predictions = [round(value) for value in y_pred]
    accuracy = accuracy_score(Test_Labels, predictions)
    print("Random Forest accuracy: %.2f%%" % (accuracy*100.0))

    return clf

rclf = random_for()

solved the problem.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.