# Why Huber loss has its form?

Huber loss formula is

$$\hspace{3.0cm} L_\delta(a) = \begin{cases} \frac{1}{2} a^2 && |a| \leq \delta \\ \delta (|a| - \frac{1}{2} \delta) && |a| > \delta\end{cases}$$ where $$a = y - f(x)$$

As I read on Wikipedia, the motivation of Huber loss is to reduce the effects of outliers by exploiting the median-unbiased property of absolute loss function $$L(a) = |a|$$ while keeping the mean-unbiased property of squared loss function $$L(a) = a^2$$ within the typical region (i.e. region without outliers)

I don't know why don't Huber loss has the form:

$$\hspace{3.0cm} L_\delta(a) = \begin{cases} \frac{1}{2} a^2 && |a| \leq \delta \\ c |a| && |a| > \delta\end{cases}$$ where $$c$$ is some constant which controls the error magnitude of outliers

or similar forms ?

Let's check for continuity for the new proposed function. For convenience, let me call it $$L_{\delta, c}$$.

$$\lim_{a \to \delta^+}L_{\delta, c}(a)=c\delta$$ and $$\lim_{a \to \delta^-}L_{\delta, c}(a)=\frac{\delta^2}2$$

If we want the function to be continuous, we would want $$c$$ to take value $$\frac{\delta}2$$.

Now, let's check for diffentiability of $$L_{\delta, \frac{\delta}2}$$.

$$\lim_{h \to 0^+} \frac{L_{\delta, \frac{\delta}2}(\delta+h)-L_{\delta, \frac{\delta}2}(\delta)}{h}=\lim_{h \to 0^+} \frac{\frac{\delta(\delta+h)}2-\frac{\delta^2}2}{h}=\frac{\delta}2$$

$$\lim_{h \to 0^-} \frac{L_{\delta, \frac{\delta}2}(\delta+h)-L_{\delta, \frac{\delta}2}(\delta)}{h}=\lim_{h \to 0^-} \frac{\frac{(\delta+h)^2}2-\frac{\delta^2}2}{h}=\delta$$

which shows that we can't have $$L_{\delta,c}$$ that is differentiable.

Now, let's examine the differentiable of the original huber loss at $$\delta$$.

$$\lim_{a \to \delta^+}L_\delta (a)=\delta(\delta-\frac{\delta}2)=\lim_{a \to \delta^-}L_{\delta}(a)=L_{\delta}(\delta)$$

Also,

$$\lim_{h \to 0^+} \frac{L_{\delta}(\delta+h)-L_{\delta}(\delta)}{h}=\lim_{h \to 0^+} \frac{\delta(\frac\delta2+h)-\frac{\delta^2}2}{h}=\delta=\lim_{h \to 0^-} \frac{L_{\delta}(\delta+h)-L_{\delta}(\delta)}{h}.$$

The huber loss function is constructed such that it is differentiable.