I'm doing multi-label classification on text data using BERT model. Since the dataset is huge, around 50 thousand rows, I was thinking to use stratify sampling on dataset to reduce it to around 2-4 thousand to hyperparameter tune on. I'm confused between trading off number of trails with size of sample set. Example: Would training 3000 rows with 5 trials will be better than training 1500 rows with 10 trials? Moreover, thinking if I should drop epoch from tuning and focus on learning rate and weight decay.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.