I'm a beginner in Machine learning and i'm searching for some optimizer for the gradient descent. I've searched many topics about that, and did a state of art of all these optimizers. I have just one problem, and i can't figure it out. Don't judge me please, but i would like to know ?

Are we using ADAM optimizer alone or are we obliged to combine it with the SGD ? I didn't understand if it works alone or if it's here to optimize NOT the neural network but the SGD of the neural network?

Thank you for your help!

  • 1
    $\begingroup$ ADAM is a standalone optimizer, so no need to be combined with SGD. It is kind of advanced flavor of classical SGD. Have you seen this post: machinelearningmastery.com/…? I think it has all you need to know about ADAM, and it is even compared with others. $\endgroup$ – TwinPenguins Aug 12 '18 at 18:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.