How to prove Softmax Numerical Stability?

Question

I was playing around with the softmax function and tried around with the numerical stability of softmax. If we increase the exponent in the numerator and denominator with the same value, the output of the softmax stays constant (see picture below where -Smax is added). I cannot figure out how to prove this numerical stability (although I read that it is true). Can anyone help me with the proof?

Answer 1

Take into accout that $e^{a-b} = e^a \cdot e^{-b}$, therefore:

$\dfrac{e^{s_{y_i} - s_{max}}}{\sum e^{s_k - s_{max}}} = \dfrac{e^{s_{y_i}} e^{- s_{max}}}{\sum e^{s_k} e^{- s_{max}}} = \dfrac{e^{s_{y_i}} e^{- s_{max}}}{e^{- s_{max}} \sum e^{s_k}} = \dfrac{e^{s_{y_i}}}{\sum e^{s_k}} $