# Equation related to Smoothness

If you have a differentiable function $$f:\mathbb{R}^d\rightarrow\mathbb{R}$$ that is $$\beta$$-smooth (for all $$v$$ and all $$w$$, you have $$\|\nabla f(v)-\nabla f(w)\| \leq \beta \|v-w\|$$), how can you show the equation below?

$$f(v) \leq f(w) + \langle\nabla f(w), v-w \rangle + \frac{\beta}{2}\|v-w\|^2$$

## 2 Answers

Let $$\gamma_t = w + t(v-w)$$. We have $$f(v) - f(w) = f(\gamma_1) - f(\gamma_0) = \int_0^1 (f \circ \gamma)’(t)dt \\ = \int_0^1 \langle \nabla f(\gamma_t), \gamma’(t)\rangle dt = \int_0^1 \langle \nabla f(\gamma_t), v-w\rangle dt \\ = \langle\nabla f(w), v-w\rangle + \int_0^1 \langle \nabla f(\gamma_t) - \nabla f(w), v-w\rangle dt.$$ By Cauchy-Schwarz and your inequality $$\langle \nabla f(\gamma_t) - \nabla f(w), v-w\rangle \leq \lVert \nabla f(\gamma_t) - \nabla f(w) \rVert \lVert v-w\rVert \\ \leq \beta \lVert \gamma_t - w\rVert \lVert v-w\rVert = \beta \, t \, \lVert v-w\rVert^2$$ so that $$\int_0^1 \langle \nabla f(\gamma_t) - \nabla f(w), v-w\rangle dt \leq \beta \, \lVert v-w\rVert^2 \int_0^1 t dt = \frac \beta 2 \lVert v-w\rVert^2$$ which proves your inequality.

• Thanks, this really is helpful! – learning machine Aug 28 '19 at 8:20

Have you tried using the mean value theorem for integrals? You can relate $$f(v)-f(w)$$ to $$\nabla f$$ and use some simple estimates to get something like the above.

• Thanks for the advice! – learning machine Aug 28 '19 at 8:20
• It's essentially is the same as the other answer, but in general when relating $f$ and $\nabla f$ it's best to always look for an integral relation. – bbbbbb Aug 28 '19 at 15:32