0
$\begingroup$

Paper on Adam mentions Stationary Objective function. I am not able to find its definition on Internet (or may be it's there with some other name and I am not able to figure it out). I will be grateful If someone can describe or direct me to the right resource.

$\endgroup$

1 Answer 1

0
$\begingroup$

A stationary point of a function is typically a local minimum or maximum, i.e. the gradient is zero, think of a parabola’s minimum. A stationary function would be a function that is constant wrt. some partial derivative. So for an objective function for a neural network, the cost or objective function is zero when the minimum is found. I believe that when they refer to a stationary objective function in the paper, they mean a function whose partial derivative with respect to the parameters of the network is constant.

$\endgroup$
1
  • $\begingroup$ Thanks @brewmaster321 . Please allow me sometime to check what you have mentioned whether it fits with author's analysis. If it does, I will accept your answer. Thank you. $\endgroup$ Commented Mar 16, 2023 at 3:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.