I am learning about different input-vector representations for Neural Networks
One of the alternatives to sparse One-Hot encoded vector is the Multi-Hot encoding.
Do I understand correctly that a traditional binary approach to counting numbers is exactly what the Multi-Hot is? We can imagine a byte as a vector of 8 components, and each entry is either 0 or 1.0
Second part of the question:
I would like to use Multi-Hot encoding to express up to 255 possible input values. If I use the binary-approach, will this be identical to Label-encoding? In a sense that network will figure out that 00000010 is "superior" to 00000001, that there is a "strong correlation and precedence"?
Or is it less exaggerated? For example, in Label-encoding, I could merely use 1 input neuron, just with a varied strength of 0 to 255, much like an enum. There the effect would be really exaggerated, as it presents a really obvious precedence of value $A$ over value $A-1$
This means I can't use it for 255 distinct categories (or for relatively unrelated categories)
Is the effect as bad in Multi-Hot encoding, in particular in a binary approach?
If it's the same as Label-Encoding (which only uses 1 input neuron), why would people ever consider Multi-hot, bloating the dimension of the input-vector?
This post points out we can use multi-hot when the input should contain $N$ concatenated one-hot vectors. For example to represent $N$ entities each of which can belong to $Z$ distinct categories. Is there also another use?