1
$\begingroup$

I am currently working on a regression problem where I have one variable (x) of the data in the form of "histogram bins". I.e. I could have value ranges 900-999, 1500-1599 etc. However the data does not tell you the specific value.

My question is: In this situation, should I treat this variable as real-valued (maybe take the median of each bin)? Or should I treat it as categorical data with each 100-wide bin representing a separate category? If I do treat it as categorical, what would be the best encoding (Label, 1-hot, etc.)?

My confusion comes from the fact that even though the data as presented is categorical, it is morally a real-valued variable. I also have prior knowledge that my target variable y should have a positive correlation with x. So if I just went with an arbitrary encoding, would it be able to capture this correlation?

Improve this question
$\endgroup$
2
  • $\begingroup$ What type of model do you use? Linear regression, random forest etc.? $\endgroup$
    – Peter
    Sep 4, 2021 at 20:31
  • $\begingroup$ I'm trying all the most common models (linear regression, random forest, knn etc.) and comparing them. $\endgroup$
    – Jack Ding
    Sep 4, 2021 at 20:33

2 Answers 2

Reset to default
1
$\begingroup$

In a linear regression, "one hot" encoding (aka "dummies") would introduce a own intercept term for each of the bins while treating the variable as numeric would introduce one (linear) slope coefficient for the bins. Treating the variable as "one hot" in a linear model will be more flexible (and will capture the positive correlation as well) since there is no forced (linear) parameterization.

However, in other model types, such as random forest (which is non-parametric), "one hot" will not necessarily work well (or better), i.e. if each single category of the one hot encoded variable has little explanatory power. So treating the variable as numeric (e.g. by median value) may work better in this case.

Basically, "one hot" encoding and numeric encoding transports pretty much the same information. The question here seems to be which model type can digest the one or the other encoding better. So if you can afford it, I would suggest trying both (one hot and numeric encoding) in different models.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the answer! After some more exploratory data analysis I realized that not all bins were of the same width. I'm not sure if that makes a difference. I'll do some more experiments to confirm. $\endgroup$
    – Jack Ding
    Sep 4, 2021 at 21:02
  • $\begingroup$ Probably a good argument in favor of one hot $\endgroup$
    – Peter
    Sep 4, 2021 at 21:04
  • 1
    $\begingroup$ Just tried it. Taking the median seems to work much better, especially on ensemble models. $\endgroup$
    – Jack Ding
    Sep 4, 2021 at 21:21
0
$\begingroup$

I think it would be better if you works directly with histograms in the framework of symbolic data analysis. You can use the HistDAWass package for R that gives you several tools to work with this kind of data.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.