Is it possible to apply a custom loss function in a regression model (or any other algorithm for predicting continuous variable) ? I'm working on a stock market prediction model and I need to maximize the following loss function: if [predicted] < [actual] then [predicted] else [-actual]. Would that be possible ? Thanks
2 Answers
The concept of a loss function comes from decision theory. Although there are some 'classic' loss functions the point is to be subjective, in the sense of being flexible enough to represent any particular problem conctext. So in that sense, yes, loss functions can be customised. One of the main ways this has been achieved is via Bayesian regression, as the output of Bayesian regression is a probability distribution which can then the input for a loss function. This blog post shows the mechanics in a simple way (texts for Bayesian statistics and decision are often non-accessible), and also gives some examples of somewhat 'custom' loss functions in 'Examples: Part 3'. -> http://www.statsathome.com/2017/10/12/bayesian-decision-theory-made-ridiculously-simple/
For most optimization algorithms, it is desirable to have a loss function that is globally continuous and differentiable.
Two very commonly used loss functions are the squared loss and absolute loss. However, the absolute loss has the disadvantage that it is not differentiable at 0. The squared loss has the disadvantage that it has the tendency to be dominated by outliers
In your case - You must convert your need into a function with the above property.
In addition to above point, another point to check -
You current loss function will not follow the correlation i.e. "Lesser Loss => Good prediction".
e.g. y_pred = 500, 1000, 2000 and y_pred = 499, 999, 1999.
Prediction is accurate still your loss will be very high i.e. 499, 999, 1999
-
$\begingroup$ How can I possibly convert my if statement to a continuous and differentiable function ? $\endgroup$– MattMay 31, 2020 at 23:31
-
$\begingroup$ Also, I don't insist on applying regression to this problem. Can it be handled by some other algorithm ? $\endgroup$– MattMay 31, 2020 at 23:40