ADAM is an optimization algorithm that is often used in deep learning. It is a stochastic gradient descent algorithm that is based on adaptive learning rates. It combines the ideas of momentum and RMSprop, which are both methods that help accelerate the convergence of stochastic gradient descent.
Momentum is a method that helps accelerate the convergence of SGD by adding a fraction of the previous update to the current update. This fraction is called the momentum coefficient, and it is typically set to a value between 0 and 1. Momentum helps the optimizer "roll over" any local minima and helps it converge to the global minimum faster.
RMSprop, on the other hand, is a method that helps improve the convergence of SGD by using a moving average of the squared gradients to scale the learning rate. This helps the optimizer avoid oscillating or getting stuck in local minima.
ADAM combines these two methods by using exponential moving averages of the gradients and the squared gradients to scale the learning rate. This allows ADAM to adapt the learning rates of each parameter individually, which can help the optimizer converge faster.
To answer your question, if you set $\beta_{1}=0$ in ADAM, it will stop using the momentum term and will only use the RMSprop term. This means that it will behave exactly like the RMSprop optimizer. Similarly, if you set $\beta_{2}=0$ , it will stop using the RMSprop term and will only use the momentum term. This means that it will behave exactly like the Momentum optimizer.