I am using the BERT model in order to classify stereotypes in sentences. I wanted to know if there is a way to automate the optimization of hyperparameters such as 'epochs', 'batchs' or 'learning rate' with some function that is similar to 'GridSearchCV' (I don't know if this function can be used in the BERT model, if it can be used let me know) so I don't have to test combinations of values 'by hand'.

I attach part of my code where I declare the hyperparameters in case there is a function that automates the search for better hyperparameters, someone could tell me where to include it.

from sklearn.model_selection import train_test_split
df_train, df_val = train_test_split(df_detests, test_size=0.2)
BATCH = 16
model = BETOClassifier()
LR = 1e-6
train(model, df_train, df_val, LR, EPOCHS, BATCH)
evaluate(model, df_test, BATCH, False)

I would also like to know if it is possible to extract metrics from the BERT model such as f1 score



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.