3
$\begingroup$

I have a regression problem where a neural network has to predict a value from $0$ to $19.999...$. I would like to convert this regression problem into a classification problem with the classes $0, 1, 2, ..., 18, 19$. Therefore I am using a $\mathrm{softmax}$-output layer.

Now, if I use a categorical crossentropy-loss, the loss is the same if the network classifies an input as $1$ instead of $14$ and if the input is classified as $13$ instead of $14$. I like to punish the loss more if the predicted value / class is farther away from the true value / class.

Is there a better loss function than categorical crossentropy for this use-case?

$\endgroup$
1
  • 1
    $\begingroup$ You could do a weighted categorical crossentropy, but why are you specifically turning it into a classification problem? $\endgroup$ – Jan van der Vegt Oct 4 '17 at 15:17
3
$\begingroup$

I think what you should be looking for is ordinal regression/classification in order to utilize the ordering of the classes.

Example python/matlab implementations, with some extra resources.

For fast integration with your system, you could create a new ordinal regression model using the outputs of the already trained network as input to the new ordinal model and the true ordinal value as the output.

For another quick/dirty solution you could try simple Mean Absolute Error or Mean Squared Error, according to this study, between the predicted and target classes of your system, as it is right now, instead of the categorical cross-entropy error function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.