6
$\begingroup$

Continous feature discretization usually leads to lose of information due to the binning process. However most of the Top solutions for Kaggle Titanic are based on discretization(age,fare).

When should continuous features be discretized ? Is there any criteria and pros and cons on accuracy.

Improve this question
$\endgroup$
2
  • $\begingroup$ When the discretization captures most of the information. It is like asking when you should do dimensionality reduction. $\endgroup$ – Emre Aug 16 '17 at 20:51
  • $\begingroup$ In the Kaggle Titanic competition discretizing passenger ages may make sense, e.g. one can divide into <5, 5 to 60, > 60 without losing information. One can argue that < 5 can't swim, and that health of > 60 make them unlikely to survive cold weather/water temperature. By the same token, fares may relate to passenger classes in some way, and one may then define arbitrary fares thresholds up to class 1 (as many thresholds as the number of classes), arguing that class 1 passengers are more likely to survive (maybe their cabins are on higher decks and closer to lifeboats) $\endgroup$ – tagoma Sep 16 '17 at 20:47
5
$\begingroup$

One reason to discretize continuous features is to improve signal-to-noise ratio. Fitting a model to bins reduces the impact that small fluctuates in the data has on the model, often small fluctuates are just noise. Each bin "smooths" out the fluctuates/noises in sections of the data.

Improve this answer
$\endgroup$
1
$\begingroup$

I can think of three reasons why discretization might help in some problems.

It makes sense for your problem

Continuous variables such as age are better understood when discretized into meaningful groups: infants, youngsters, young adults, adults, senior, ... this is common in the field of marketing, because a small number of years do not really make much different in one person's interests.

To give another example, when working on a dataset with GPS locations, it might be more useful to discretize those into contry/state locations.

Interpretability

A continuous feature might not linearly correlate with your target but have a more complex non-linear correlation. In that case, obtaining an interpretable explanation of such feature won't be easy. However it you discretize it into a set of groups or levels, you might find that some of them correlate (or anticorrelate) with your target, giving you some interpretability.

Model limitations

Some machine learning models and feature selection methods can't handle continuous features, such as entropy-based methods, or some variants of decision trees or neural networks. Either you discretize your features or forget about using such models.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Would need an answer respect to ROC/AUC and metrics . Most of the algorithms works on both sets of data . Assuming algorithms working on both what is the quantifiable rational reason to use discretization especially when it leads to lose of information. Accuracy increases or decreases with same algos ? $\endgroup$ – drichlet Jun 18 '17 at 4:08
  • $\begingroup$ As far as I know there are no proofs for this in terms of ROC/AUC. My intuition tells me you can't prove that, as indeed generally you will lose information through discretization, but you might actually get better results, as discretization itself could act as a form of regularization, avoiding overfitting to arbitrary continuous values. $\endgroup$ – albarji Jun 25 '17 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.