0
$\begingroup$

I have read quite a lot about discretization techinques, measuring WoE and IV etc., but the basic question - when is it worth to use binning and when isn't - is not entirely clear. On the one hand, the result of discretization seems quite "pretty" and more easy to handle for me, but on the other hand it is always emphasized that binning always causes information loss. Could you provide some clues when should one use this technique?

$\endgroup$
1
$\begingroup$

Mostly this is avoided if reasonably possible in my experience, but:

One scenario where you may want to consider binning is if you are doing modeling with a logistic model. The coefficients indicate that per unit increase of the IV, the log odds of the dependent variable will increase by ___. The important distinction is that we are assuming a linear relationship in the logit by saying "per unit increase, we see ___ increase", when in fact it may not be linear. To fix this, one possible solution is two bin/discretize your IV to make the relationship more linear, and this interpretable. This is only important if you are interested in interpretability, not just prediction. There are also other techniques to account for this that may be preferable (like generalized additive models)

Another scenario is for certain kinds of "big" data where the signals can shift a little bit on an axis due to measurement/machine error, and binning techniques allow you to make sure that the correct signal is matched up closely to other runs where the signal may be in a slightly different position. like below: you may bin 65,000 x-axis values into 3200 x-axis values and average their appropriate y-values. This reduces dimensionality and accounts for the minor shifts in x-axis position that can occur. e.g. bin 1324 will likely capture the same signal across all other runs, where if you specificed bin 23,950, it might correspond to bin 24,000 on another run of the machine/technique, and your columns wouldn't match up correctly between samples. In the example below, by binning you are making sure each respective compound is correctly captured, considering slight x-axis variation.

enter image description here

Last but not least, among several other factors seen here and here, you are purposefully dropping information for the model, which means you will lose power, as you mentioned. Also, when you are deciding a cutpoint to (for example) categorize a continuous variable as high vs low, how do you choose the cutpoint? Can you defend your choice, or is it somewhat arbitrary? Did you choose multiple cutpoints to see which one favored your results and thus committed the tempting garden of forking paths/multiple testing?

$\endgroup$
1
  • $\begingroup$ Great response. I always emphasize these problems. Most importantly, in the worst case scenario, you can end up giving up all of your information if you choose these apriori. It's usually better to let a learning algorithm identify the cut points in a supervised way (e.g., MARS). $\endgroup$ – franciscojavierarceo Nov 26 '16 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.