1
$\begingroup$

In the case I want to predict only ranges from a continuous value, is there any reason to use regression instead of classification ? Could it depend on the type of model I am using (neural network, decision tree, bayesian, ...) ?

Example

Let say I have a dataset with images. Each image has one human on it and is labeled with his/her height. Now I am only interested in predicting height ranges, for instance these four classes [ A, B, C, D ] = [ <150, 150-170, 170-190, >190 ] (in cm). Is there any reason why one of the two following approaches should lead to better performances ?

  • case 1: using regression - First create and fit a model that predicts the exact height from an image, then simply gives its associated height range.
  • case 2: using classification - First label all the images with the wanted ranges (=classes), then create and fit a classifier to predict this height range.

Note: I am wondering if there is a general answer to this question, not only to this example

EDIT

As @n1tk pointed out, in the post Performance of CNN based deep models with number of classes, the question is answered if we think about increasing the number of classes. In my question, I am wondering about regression vs classification. So try to fit a continuous value vs ranges from this value.

Improve this question
$\endgroup$
12
  • 2
    $\begingroup$ Does this answer your question? Performance of CNN based deep models with number of classes $\endgroup$
    – n1tk
    Sep 5 '20 at 1:52
  • $\begingroup$ In the post you suggest, it seems the question is about the number of classes, so it is is only about classification, am I wrong ? So if a neural network is used (as in the post), one hot encoded vectors would probably be used for the labeling and the backpropagation. Here I was more wondering about regression vs classification. It might be related but it is not obvious to me. $\endgroup$
    – etiennedm
    Sep 5 '20 at 16:53
  • $\begingroup$ stackoverflow is full will comparison of logistic vs OLS. You point as interval there and is nothing else than a category because you predict [A, B, C, D] ... one way is to transform Y with logit and fit GLM. here is another post related to logit vs OLS on time series data stats.stackexchange.com/q/201299/80330 $\endgroup$
    – n1tk
    Sep 5 '20 at 17:09
  • 1
    $\begingroup$ to answer better your question look at assumptions of each regression and will point in right direction ... going to what you try to do: you look for probability of an event to be in that range vs actual continuous value with OLS ... and going to task in CNN you are looking for prob of an event when you look at activation function for neurons (perceptron and logit info will help to understand first link) $\endgroup$
    – n1tk
    Sep 5 '20 at 17:14
  • $\begingroup$ I see your point and I agree. So both should lead in the same performances. Thank you for your answer $\endgroup$
    – etiennedm
    Sep 5 '20 at 17:16
1
$\begingroup$

The general answer is how the model will be used deal. Either way may be optimal for the case.

For example - If the model groups applicants into good credit risk and bad credit risk, that might be fine to say model score > x = good risk and model score <= x = bad risk. But maybe there will be differential action based on the model score - like giving a different interest rate or a bigger loan.

In the original example, in regression actual = 191, predicted = 189 you can calculate the loss.

In classification, if actual = 191 and P(>190)=0.35, P(170-190)=0.40, P(150-170)=0.25, then you just know the wrong class. Is that enough for the model usage?

There is also the assumption that a "closer" class will be chosen but that might not be true, e.g actual=191, P(>190)=0.25, P(170-190)=0.25, P(150-170)=0.5. The regression could come up with 160 also but you can measure that loss if the model usage requires it. Many classification algorithms do not know if classes are "close" - Confusion matrix. "How close I am to the diagonal?". Is there such metric?

You can also look at Ordinal Regression https://en.wikipedia.org/wiki/Ordinal_regression. In this case there is an implicit ranking in the "class".

Choose based on how the model will be used. Always important to know the usage and the problem being solved, then work backwards to the model.

Hope that helps.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks for your answer ! You made a good point regarding the bounds of the ranges. However, I think this could be managed by increasing the classes resolution and/or by using overlapped ranges depending on how the model will be used, don't you think ? Regarding ordinal regression, I don't understand the link with the question, could you please detail ? $\endgroup$
    – etiennedm
    Sep 4 '20 at 11:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.