I have written code for binary text classification using XLM-RoBERTaForSequenceClassification. My train_dataset is made up over 10.000 data. For training I have used a batch size=32. The text hasn't been cleaned too much (I removed tickers, number, lowercase, hyperlinks, hashtags, words with 2 or fewer letters, words with 2 or fewer letters, words with 2 or fewer letters, emoticon) but I get overfitting after only 10 epochs. My question is, if I increase the batch size it is possible to "avoid" overfitting?
-
1$\begingroup$ Nobody can tell, but probably not, maybe add a dropout rate or regularization. $\endgroup$– user2974951Jun 9 at 10:08