2
$\begingroup$

I am training a Squeeze-net model for binary classification of images. I have 79968 images for training (50:50 for and against) and 8892 images in the validation set. After 35000 iterations my training accuracy fluctuates between 1 and 0.96875. The validation accuracy is more or less constant between [0.985, 0.986]. The base learning rate is 0.01 and decreased upto 0.00001. As far as I can tell visually training loss doesn't exactly fluctuate between two numbers as such but is in the range [0.02, 0.09] for most part except a few occasional spike.

My question is what can i infer from this? (1) Overfit? (2) Model has converged? If not should I reduce the learning rate? (3) Is the model stuck at a local minima? I am using softmax-with-loss as my loss layer.

$\endgroup$
1
  • $\begingroup$ can you provide the accuracy and loss plots? $\endgroup$ Sep 24, 2018 at 14:40

1 Answer 1

1
$\begingroup$

Did the loss decrease up to a certain point and then start fluctuating between .02 and .09? If so you may have just reached your convergence and now it's oscillating due to the size of the learning rate being too large in relation to the size of the minima you have hit. Reduce your learning rate and see where it ends up, my guess is that you will get close to .02 this time around, unless you have truly hit a local minima, in which case you may go lower.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.