I have a few basic questions about tracking losses during training.
- If I am using mini-batch training, should I validate after each batch update or after I have seen the entire dataset?
- What should be the condition to stop the training to prevent overfitting? Do you save the model at that point?
- In case I use mini-batch training losses fluctuate a lot, depending on the random choice of training data, and sometimes validation loss is less than training loss. Is this normal? I think my confusion about this point will be answered in 1 by itself.