I'm working with a really high-cardinality feature as one of the inputs to my model and I'm using hash-encoded feature embedding rather than one-hot encoding. However, this method is ignoring the frequency of each category in each sample.
As an analogy - Imaging representing a document as an embedding of list of topics. Each topic has a certain score associated representing how strongly it appears in the document. The possible list of topics is large.
An option for a smaller cardinality would be to have a multi-hot list where each index represented a category and then value would represent a score:
[0 0 0.1 0 0.2 0 ...]
0 1 2 3 4 5 ...
However, the dataset a really high cardinality (~100M). I'm looking for ways I can use a list of both - categories and corresponding values and create a low-dim embedding.
