3
$\begingroup$

Are there any reasons to turn regression into classification - binning continuous targets into classes and then learning model on those instead? (Let's say the algorithm isn't a problem.)

$\endgroup$

2 Answers 2

2
$\begingroup$

You actually convert the output of your algorithm from continuous to categorical.

I see many reasons of why you would want to do that. A simple case for this would be when you have very long time-series data that take a lot of space to be saved. In this case, it is convenient to convert the timeseries in a histogram representation, which actually is a number of predefined bins where all the values fall into. This will greatly reduce the memory requirements of your algorithm because you just increase the count of a bin (integer number) when a measurement is taken that falls in it, instead of storing the measurement itself as a float number.

This of course has pitfalls, such as losing the "time" dependencies between the measurements as well as reduction in resolution of your dataset. BUT, it can be really useful when forecasting measurements because instead of having the continuous R space as output of a predictor, you actually have (e.g.) 10 bins (classes) that the next measurement is predicted to fall into, making it easier for your model to be trained.

FYI, this is exactly what is done in the automotive industry.

EDIT - (Added sources)

Sources:

https://people.dsv.su.se/~tony/papers/dmin_2015.pdf (check the algorithm)

https://www.phmsociety.org/node/2284 (the previous algorithm is applied here for trucks)

https://link.springer.com/article/10.1007/s10618-017-0538-6 (another algorithm based on histograms of truck data)

$\endgroup$
2
  • $\begingroup$ Do you have any examples/use cases/references from automotive industry you mentioned this strategy is used? $\endgroup$ Jun 2, 2018 at 11:23
  • $\begingroup$ sure, I edited my answer please check :) $\endgroup$
    – pcko1
    Jun 2, 2018 at 11:37
0
$\begingroup$

While doing this might simplify your analysis, it is not a recommended approach.

Let’s use an example. Suppose that you are using a series of explanatory variables (X1, X2, X3) to estimate car sales (in US$).

Now, suppose that our variable for car sales is interval, i.e. we have a series of car sales figures (25000, 37500, 3000, 71000…), etc. Let’s say you were to convert your dependent variable to a categorical one. e.g. >25000 = 1, <25000 = 0.

By doing this, you would lose a lot of information from your dependent variable and your model would have significantly less capacity to quantify unit effects of each explanatory variable on the dependent one.

This is why when binary logistic regressions are run, it is often recommended that a minimum of 500 observations should be used to induce a significant level of variation in the dependent variable to analyse the effects of such variation (Studenmund, 2010). Additionally, it is for this reason that traditional measures of fit in a regression model such as R-Squared become invalid when analysing a dataset with a categorical dependent variable.

Categorical variables – or classes – are used in situations where it is not possible to use an interval variable to quantify a particular condition. e.g. suppose you are a medical researcher constructing a model to determine the presence of diabetes in a particular patient. Now, either a person has diabetes or he does not (diabetes = 1, no diabetes = 0). Therefore, your model is not “losing” information because the dependent variable is providing all the information necessary for processing the model.

By discarding information, you risk increasing the possibility of a Type 1 error – where you reject a true null hypothesis.

In summary, it depends on the data you are working with. In some instances, the dependent variable might make more sense if it were expressed categorically. However, there are many other instances where it is not a recommended approach.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.