2
$\begingroup$

I’m working on a regression problem in which I’d like to predict demand of different items. I have used holidays as a feature in my model, in a hot encoded format, i.e. I have 11 binary features each representing one holiday.

I’d like to reduce the size of my features and thinking to used embedding on the holiday features to represent them in lower dimensions.

I’m new to embedding. My question whether it make sense or not and if it does any hints on how to do it?

$\endgroup$
2
$\begingroup$

Doing it is easy. Simply create a mapping between your 11 values and embeddings of any size. Choosing the values for the embeddings is typically done through training a neural network that embeddings are part of. You could use dimensionality reduction techniques like PCA for instance as an alternative.

Now, embeddings only make sense if they represent a feature of dimensionality $D$ with embeddings of dimensionality between 1 and $D-1$. So in your case, I don't know how much you will benefit from the dimensionality reduction itself. You may experience benefits of replacing sparse with dense features, but this needs to be empirically tested.

$\endgroup$
2
  • $\begingroup$ my values are holiday flags, so each observation within my training data would have only one these flags set as 1. having said that, is embedding useful here? $\endgroup$
    – HHH
    Jan 14 '21 at 18:13
  • $\begingroup$ I would say no. 11 features is not that many, so I doubt you'll get huge benefits, but of course, it would need to be empirically proven. Embeddings could become more useful if you enrich your flags with other info regarding how long the holidays are, in what month they fall, are they religious/yes/no. $\endgroup$ Jan 14 '21 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.