# What are some good loss functions used to minimize extreme errors in regression and time series forecasting?

E.g. In detriment of a smaller mean error, I want to have fewer big mistakes

I'm working on a time series forecasting task and in some specific cases I don't need perfect accuracy, but the network cannot by any mean miss by a lot.

Any suggestions of loss functions or other methods to solve this issue?

## 2 Answers

As you increase the harshness of big misses, you make the model less willing to miss big.

For instance, absolute loss considers missing by $$2$$ to be twice as bad as missing by $$1$$, but square loss considers missing by $$2$$ to be four times as bad as missing by $$1$$!

Imagine if you use cubic loss:

$$L_3(y,\hat{y}) = \sqrt{\sum\big\vert y_i - \hat{y}_i \big\vert^3}$$

Imagine if you use quintic loss:

$$L_5(y,\hat{y}) = \sqrt{\sum\big\vert y_i - \hat{y}_i \big\vert^5}$$

This gets at the general form of $$L_p$$ loss, which I'm guessing you can guess before reading what I type.

$$L_p(y,\hat{y}) = \sqrt[p]{\sum\big\vert y_i - \hat{y}_i \big\vert^p}$$

This relates to the idea of $$L^p$$ norms.

At the extreme of $$L^p$$ norms, you can set $$p=\infty$$, which is equivalent to just looking at the maximum. If you use $$L_{\infty}$$ loss, you minimize the greatest error.

I've wanted to play around with $$L_{\infty}$$ loss for a while and would be very curious what happens if you choose this loss function. I do, however, think this would be a very extreme loss function.

(Remember that $$RMSE$$, $$MSE$$, and $$SSE$$ all have the same $$argmin$$; by similar logic we are free to take the $$p^{th}$$ roots to get the cubic and qunitic loss functions I gave.)

You could consider something like the relative error $$L(y,\hat{y}) = \sqrt[]{\sum\big\vert (y_i - \hat{y}_i) / y_i \big\vert^2}$$