0
$\begingroup$

I know it's easy to do grid search for a simple Catboost model, such as in here: https://medium.com/aiplusoau/hyperparameter-tuning-a5fe69d2a6c7

by running something like

cbc = CatBoostRegressor()

#create the grid
grid = {'max_depth': [3,4,5],'n_estimators':[100, 200, 300]}

#Instantiate GridSearchCV
gscv = GridSearchCV (estimator = cbc, param_grid = grid, scoring 
='accuracy', cv = 5)

#fit the model
gscv.fit(X,y)

#returns the estimator with the best performance
print(gscv.best_estimator_)

Method like this did not have the input of categorical columns in the Catboost model.

But my question is how can I do grid search for categorical_cols specified?

For example, here is my code how I assign the categorical columns:

categorical_cols =  ['site_number','product_key', 'manufacturer_desc']


# initialize Pool
train_pool = Pool(X_train, 
                  y_train, 
                  cat_features=categorical_cols)
test_pool = Pool(X_test, 
                 cat_features=categorical_cols) 


# specify the training parameters 
model = CatBoostRegressor(iterations=150, 
                          learning_rate = 0.5,
                          depth=8,  
                          random_seed = 42
                          )
#train the model
model.fit(train_pool)

But this is the model without grid search. The question is how can I still do the grid search with the above categorical_cols specified. The train_pool and test_pool is already specified, not sure what's a best way.

Thanks!

$\endgroup$

1 Answer 1

0
$\begingroup$

To perform a grid search with specified categorical columns in CatBoost, you can use the GridSearchCV function from Scikit-learn. You can define a parameter grid with different values for the hyperparameters you want to tune, including the categorical columns. Here's an example:

from catboost import Pool, CatBoostRegressor
from sklearn.model_selection import GridSearchCV

# define the parameter grid
params = {
    'iterations': [100, 150, 200],
    'learning_rate': [0.1, 0.5, 1],
    'depth': [6, 8, 10],
    'cat_features': [['site_number', 'product_key', 'manufacturer_desc'],
                     ['site_number', 'product_key'],
                     ['product_key', 'manufacturer_desc']]
}

# initialize Pool
train_pool = Pool(X_train, 
                  y_train, 
                  cat_features=categorical_cols)
test_pool = Pool(X_test, 
                 cat_features=categorical_cols) 

# initialize the model
cat = CatBoostRegressor(random_seed=42, silent=True)

# perform grid search with 5-fold cross-validation
grid_search = GridSearchCV(cat, param_grid=params, cv=5)

# fit the grid search to the data
grid_search.fit(train_pool)

# print the best hyperparameters
print(grid_search.best_params_)

In this example, the params dictionary contains different values for the iterations, learning_rate, depth, and cat_features hyperparameters. The cat_features parameter takes a list of lists, where each list is a different combination of categorical columns. The GridSearchCV function performs a grid search with 5-fold cross-validation to find the best combination of hyperparameters. The best hyperparameters can be accessed with the best_params_ attribute of the GridSearchCV object (including the ones you provided in your code if they merit the best results).

$\endgroup$
2
  • $\begingroup$ Thanks! Now I got an error ' TypeError: Singleton array array(, dtype=object) cannot be considered a valid collection.' at 'grid_search.fit(train_pool)' $\endgroup$
    – Ian
    May 24, 2023 at 21:06
  • $\begingroup$ The error suggests that the train_pool object is a singleton array, which cannot be considered a valid collection. This could be due to a number of factors, such as incorrectly formatted input data or improperly specified hyperparameters. Double-check your input data to ensure that it is properly formatted, and that all necessary columns and features are included. You may want to review the hyperparameters that you have specified to ensure that they are appropriate for your data and problem domain. Consult the documentation for the CatBoost library otherwise as well. $\endgroup$
    – RegressIt
    May 25, 2023 at 7:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.