I have trained my neural network model with optimizers such as RMSProp, AdaGrad, Momentum, and Adam.
Currently, after running the code, I have printed out the Train and Test Accuracy of every epoch (50 in my case). However, I would like to know how should I determine which of these optimizers performs the best?
Does a higher train accuracy at the last epoch determine which is best or would a higher test accuracy do so? Also, I observed that when using the Momentum optimizer, the model train accuracy reached its' highest around 0.91 in the 16th epoch compared to the other optimizer.
Hence, would that conclude that the Momentum optimizer performs best in this case?