2
$\begingroup$

I have a dataset that follows Zipf's law such that the majority of the values are concentrated at one end, with the remaining items containing a very small percentage. Training on the dataset as is would introduce a bias, and thus I was thinking of restructuring the data to fall into buckets. Thus my model would be a multi-class classification model, rather than a regression model (I am training a NN).

My question is whether I should draw up the buckets such that the distribution of the items is uniform, or normal. A uniform distribution would ensure that the NN has the same amount of examples of each bucket, however, some say that normal distributions work better for machine learning. Which one should I use?

Thanks

$\endgroup$
1
  • $\begingroup$ How would training on that dataset create a bias? $\endgroup$
    – Dave
    Nov 29, 2021 at 17:09

1 Answer 1

0
$\begingroup$

From my understanding, you are describing the distribution of target (outcome) values.

There are many options. You can directy transform the target values to be a normal or uniform distribution while remaining a regression problem. One example is scikit-learn's TransformedTargetRegressor.

You can also discretize (bin) the data.

The "normal distributions work better for machine learning" typically refer to feature values, not target values.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.