When used in a linear model, a convex loss function guarantees a unique global minimum for the parameters, which can be found by local optimization methods.
However, when the model is nonlinear (e.g. MLPs), local minima are possible for a convex loss.
Are there any benefits to a convex loss function when the model is nonlinear? Can convexity be completely disregarded in the nonlinear case?