0
$\begingroup$

Let's say I have a continuous (real-valued) variable that I'll be bucketizing as part of my feature engineering. If I want to end up with the same number of items in each bucket (a flat histogram), I can take one of the following two approaches:

  1. Finding the minimum and maximum values of the variable and bucketize the range (min to max) with some predefined number of buckets. Then, I can calculate the cumulative version of the histogram and use it to equalize the data.
  2. Transforming the original continuous variable (somehow, e.g. log()) and then bucketize it. Given a well-designed transformation, the resulting buckets will be holding the same number of samples (A.K.A. flat histogram).

The first approach is what most libraries (like OpenCV) have implemented. That's because they are designed for discrete data (like pixel values) so they are perfectly sufficient. But if the original variable is continuous and in need of bucketizing, the second approach could end up with a much smoother equalized histogram. In other words, the first approach is losing data if the variable is continuous.

While I can hypothesize so far, I don't know how to come up with the transformation needed in the second approach. Does anyone know if a method exists or even better there's a library that can do this for me?

$\endgroup$
3
  • $\begingroup$ Basically, it is an equalized histogram based on log values, but with bins having different widths so that they can contain the same number of samples? $\endgroup$ Commented Dec 28, 2022 at 16:29
  • $\begingroup$ Well, the log() was just an example. Not necessarily part of the solution. But what you said kinda makes sense. Do I just need to count the total number of samples, divide them by the number of bins, and then starting from the min, call it a bin each time I reach the number of items for each bin!? So basically, no formula, what-so-ever!? $\endgroup$
    – Mehran
    Commented Dec 28, 2022 at 18:29
  • $\begingroup$ Yes indeed. I've been turning around complex solutions but it could be as simple as that. You just have to order the samples, count them until reaching the target count, and define each min and max values for every histogram bin edge. $\endgroup$ Commented Dec 28, 2022 at 19:53

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.