I need to finetune BERT model (from the huggingface repository) on a sentence classification task. However, my dataset is really small. I have 12K sentences and only 10% of them are from positive classes. Does anyone here have any experience on finetuning bert in small datasets? Do you have any suggestions for learning rate, batch size, epoch number warmup steps etc.?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.