As you increase the harshness of big misses, you make the model less willing to miss big.
For instance, absolute loss considers missing by $2$ to be twice as bad as missing by $1$, but square loss considers missing by $2$ to be four times as bad as missing by $1$!
Imagine if you use cubic loss:
$$L_3(y,\hat{y}) = \sqrt[3]{\sum\big\vert y_i - \hat{y}_i \big\vert^3}$$
Imagine if you use quintic loss:
$$L_5(y,\hat{y}) = \sqrt[5]{\sum\big\vert y_i - \hat{y}_i \big\vert^5}$$
This gets at the general form of $L_p$ loss, which I'm guessing you can guess before reading what I type.
$$L_p(y,\hat{y}) = \sqrt[p]{\sum\big\vert y_i - \hat{y}_i \big\vert^p}$$
This relates to the idea of $L^p$ norms.
At the extreme of $L^p$ norms, you can set $p=\infty$, which is equivalent to just looking at the maximum. If you use $L_{\infty}$ loss, you minimize the greatest error.
I've wanted to play around with $L_{\infty}$ loss for a while and would be very curious what happens if you choose this loss function. I do, however, think this would be a very extreme loss function.
(Remember that $RMSE$, $MSE$, and $SSE$ all have the same $argmin$; by similar logic we are free to take the $p^{th}$ roots to get the cubic and qunitic loss functions I gave.)