# How does Tensorflow compute gradients of reduce_min operation?

With a non-differentiable operation, such as a minimization, how does Tensorflow compute the gradients? Some kind of soft-minimum approximation? If so, can I retrieve the analytical computation for a specific gradient?

A minimum operation is differentiable, or at least you can easily express the partial derivatives w.r.t. its inputs:

## $\frac{\partial f}{\partial x_i} = \begin{cases} 1,& \text{if } argmin_i(x_i) = i\\ 0, & \text{otherwise} \end{cases}$

This does not hold strictly when multiple values share the same minimum value. However, that is not a problem in practice for gradient-based optimisers in TensorFlow, which can simply set all tied indices to have partial derivative of 1 (or a fraction $\frac{1}{n_{min}}$), with little impact to the eventual result, because ties for values will happen rarely. Ties may happen frequently enough in a ReLU-based network that the TensorFlow developers have considered a best response for them - I don't know specifically what TensorFlow does for that situation.

• Is there any reference to this in the Tensorflow documentation? Commented Jan 14, 2020 at 5:03
• Answer is contradictory. Says ~ "min is differentiable ... expect when it isn't ...". In a strict mathematical sense min(x,y,z,...) is not differentiable over it's domain. Commented Jan 14, 2020 at 5:41