I'm working with a really high-cardinality feature as one of the inputs to my model and I'm using hash-encoded feature embedding rather than one-hot encoding. However, this method is ignoring the frequency of each category in each sample.

As an analogy - Imaging representing a document as an embedding of list of topics. Each topic has a certain score associated representing how strongly it appears in the document. The possible list of topics is large.

An option for a smaller cardinality would be to have a multi-hot list where each index represented a category and then value would represent a score:

[0 0 0.1 0 0.2 0 ...]
 0 1  2  3  4  5 ...

However, the dataset a really high cardinality (~100M). I'm looking for ways I can use a list of both - categories and corresponding values and create a low-dim embedding.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.