Cast regression problem into classification problem

Question

In the TA session, my TA claimed, that regression problems should often be cast into classification problems by dividing the output range into bins and then using a multi-loss, since we have better classification than regression algorithms.

In my understanding, this is inherently wrong as it discards the property, that "close to correct is better than far correct". All wrong classes are equally wrong. I asked my professor, but he just said, there are applications where it makes sense and did not want to discuss it more.

Am I wrong? When should I cast a regression problem into a classification problem?

Edit: I do not know if my TA referred to it, but here is a tweet from A. Karpathy: https://twitter.com/karpathy/status/708480082831024128

not-widely-enough-known-protip: Do not use L2 loss (regression) in neural nets unless you absolutely have to. Softmax likely to work better.

Answer 1

The statement in principle seems very vague, how can one state that there are better classification than regression algorithms?

With that said I would rephrase the statement to:

Sometime it is feasible to turn a regression into a classification problem due to for the problem itself, it makes sense to predict a range/bin instead of a continuous value.

When doing this we have to be careful since we are not facing a "common" classification problem but we have an ordinal classification problem in which we have a natural order from the new target (bins)

For reference check:

https://stats.stackexchange.com/questions/493254/why-ordinal-target-in-classification-problems-needs-special-attention

Answer 2

It really comes down again to statistical modeling vs. decision-making. But I generally agree with you that the practice isn't beneficial; at the very least I think your TAs statement with the word "often" is incorrect.

---

In the TA session, my TA claimed, that regression problems should often be cast into classification problems by dividing the output range into bins and then using a multi-loss...

This seems wrong. If you use more than two bins, then the problem still should be treated as ordinal rather than flat classification.

...since we have better classification than regression algorithms.

This also seems wrong, though it's hard to prove the negative. Could you ask your TA for examples?

In my understanding, this is inherently wrong as it discards the property, that "close to correct is better than far correct". All wrong classes are equally wrong.

Exactly, and again consider an ordinal regression as an intermediate approach. But still, the raw regression is more information. However,

there are applications where it makes sense...

Now this could be true. As an example from the replies to your linked tweet, say you're modeling the temperature, but ultimately what you care about is whether you should wear a coat. The best model of the temperature will be a regression, but if you really want to tie everything up into one model, say you discretize at 5C. Now, if your regression is far off in predicting situations with temperature 40C, say as 30C, it doesn't actually hurt your decisioning. You would in fact prefer a model that is more accurate near the cutoff value. But in the other direction, 4.5C being "misclassified" as 5.5C is perhaps not what you're looking for either...

And, given so little response from the author of the tweet, I'm disinclined to take their word for it (despite their credentials).

See also:
Reducing Regression to Classification
How to convert regression into classification?