I have 2 instances of an object detection model. The only difference between these two models is the training data used:
- The first model was trained with a small training set
- The second model was trained on a larger training set than the first one
The first model was trained on the following hyperparameters:
- Number of iterations: 250k
- Batch Size: 10
- Learning Rate: warms up to 0.001 and decreases to 0.0002 after 150k iterations
Since the second model has more training data, I assumed I need to change the hyperparameters a bit. So I tried training the second model on the following hyperparamters:
- Number of iterations: 600k
- Batch Size: 10
- Learning Rate: warms up to 0.001 and decreases to 0.0002 after 400k iterations
When I measure the mAP for both models on a testing set, the first model vastly outperforms the second model.
model 1 mAP: 0.924
model 2 mAP: 0.776
This leads me to my question for you:
How would the hyperparameters (batch size, learning rate etc) change when the size of your training set increases? What factors need to be considered for this increase in training set size, in order to get the most optimal model possible?
Any and all responses will be greatly helpful. Thank you :)