12
$\begingroup$

What is the best optimizer for Convolutional neural network (CNN)?

Can I use RMSProp for CNN or only for RNN?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
9
$\begingroup$

Yes, you can use the same optimizers you are familiar with for CNNs.

I don't think that there is a best optimizer for CNNs. The most popular in my opinion is Adam. However some people like to use a plain SGD optimizer with custom parameters.

An excellent article explaining the differences between most popular gradient descent based optimizers can be found here.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I get better accuracy with default valued SGD than ADAM. I read somewhere that that tends to be the case with CNN's. I'm actually looking for a more definitive answer that does recommend and explain why one is better than the other, as I don't seem to be the only one noticing this when optimizing a CNN. towardsdatascience.com/… $\endgroup$
    – Elegant Code
    Feb 28, 2020 at 1:18
  • $\begingroup$ Here is a link to a white paper on why SGD tends to generalize better than ADAM even though the latter tends to converge faster: proceedings.neurips.cc/paper/2020/file/… $\endgroup$
    – Zexelon
    Dec 18, 2022 at 3:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.