Reducing Regression to Classification

Question

If a regression problem is reduced to classification, does minimizing the classification loss translate to minimizing regression error and hence better regression performance?

Answer 1

No, minimizing the classification loss does not translate to minimizing regression error and does not yield better regression performance.

This is a core tenant of Measure Theory. Regression measures outcome variables on ratio, interval, or ordinal scales. Classification measures outcome variables on nominal scales. You can always go from more complex scales (i.e., ratio, interval, or ordinal) to simpler scales (i.e., nominal). There are no guarantees about going back. This is technically called a non-injective mapping.

One way to think of it is that you are picking a downsampling scheme to map the data from regression to classification, and there are many possible mappings back from classification to regression. There are no guarantees on which mapping was used. Once downsampled, the data has to stay in the lower sampling space.

Answer 2

I guess you've made some buckets from your numerical target variable.

I would answer yes to your question as being in the right bucket ("minimizing classification loss") means being closer of the target value ("minimizing regression error").

Answer 3

Well, that's my understanding of it. To evaluate ML performance metrics, consider the following.

For classification, you can use.

Classification Accuracy
Log Loss
Area Under ROC Curve
Confusion Matrix
Classification Report

For regression, you can use.

Mean Absolute Error
Mean Squared Error
R^2

See the link below for all details.

https://machinelearningmastery.com/metrics-evaluate-machine-learning-algorithms-python/