0
$\begingroup$

I have a dataset with some numerical and categorical features and I am trying to apply CatBoost for categorical encoding and classification.

Since my dataset is highly imbalanced, with a large number of data samples with label 0 compared to those with label 1, I'm also trying to use SMOTE to synthesize label 1 data samples before CatBoost classification.

My code -

# train_categorical_features is a list of columns that have categorical values
train_pool = Pool(data = X,
                  label = y,
                  cat_features = train_categorical_cols)

X_enc = train_pool.get_features()
print(X_enc)
y_enc = train_pool.get_label()
print(y_enc)

smote = SMOTE()
X_res, y_res = smote.fit_resample(X_enc, y_enc)
print('Resampled dataset samples per class {}'.format(Counter(y_res)))

predictions = []
for i in range(10):
    clf = CatBoostClassifier(learning_rate=0.08,
                         depth = 10,
                         loss_function='Logloss',
                         l2_leaf_reg = 4,
                         iterations=1000,
                         task_type="GPU",
                         random_seed=i,
                         logging_level='Silent')
    clf.fit(train_pool, plot=True,silent=True)
    predictions.append(clf.predict_proba(test[inputcols])[:,1])
    print(clf.get_best_score())

I get an error on X_enc = train_pool.get_features() that says -

CatBoostError: Pool has non-numeric features, get_features supports only numeric features

My questions are -

  1. Is my approach towards applying SMOTE with CatBoost correct?
  2. I've diligently followed the catboost documentation, and I am not able to understand or fix the error I've mentioned above. Would appreciate any help.
$\endgroup$

1 Answer 1

2
$\begingroup$

The reason you are getting the error is that your pool contains categorical features. If all of your features were numerical, it would work fine.

SMOTE is irrelevant here.

$\endgroup$
2
  • $\begingroup$ Hi! Thank you for your answer! Quick clarification - is categorical encoding happening when I fit the data into the model? i.e., after clf.fit(train_pool...) $\endgroup$ Feb 5, 2020 at 14:26
  • $\begingroup$ @AishwaryaAR, Yes, I am pretty sure that it is the case. Also categorical encoding happens per split. You can find more info here: catboost.ai/docs/concepts/… $\endgroup$
    – Akavall
    Feb 5, 2020 at 18:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.