What is the best optimizer for Convolutional neural network (CNN)?
Can I use RMSProp for CNN or only for RNN?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Yes, you can use the same optimizers you are familiar with for CNNs.
I don't think that there is a best optimizer for CNNs. The most popular in my opinion is Adam. However some people like to use a plain SGD optimizer with custom parameters.
An excellent article explaining the differences between most popular gradient descent based optimizers can be found here.
-
$\begingroup$ I get better accuracy with default valued SGD than ADAM. I read somewhere that that tends to be the case with CNN's. I'm actually looking for a more definitive answer that does recommend and explain why one is better than the other, as I don't seem to be the only one noticing this when optimizing a CNN. towardsdatascience.com/… $\endgroup$ Commented Feb 28, 2020 at 1:18
-
$\begingroup$ Here is a link to a white paper on why SGD tends to generalize better than ADAM even though the latter tends to converge faster: proceedings.neurips.cc/paper/2020/file/… $\endgroup$– ZexelonCommented Dec 18, 2022 at 3:57