When can embeddings be useful for small input spaces?

Question

I've noticed that some authors tend to use embeddings for almost everything. E.g. in the AlphaStar paper, or below an example for the OpenAI5 paper (link):

The Hero Embedding here encodes information about which hero out of 17 was picked. With 5 pickable heroes per game, that would yield a one-hot encoded vector of size 17 * 5 = 85. Instead, they chose to encode this information using an embedding. Does that not seem a bit overkill?

My understanding was that embeddings are very useful for large scale inputs, such as word-corpora, as they reduce the dimensionality nicely. What's the benefit of doing it on smaller inputs such as above? Is it just convenience? Or maybe I am misunderstanding what embedding means here?

EDIT: I wanted to add that EVEN if the Hero Embedding contains more unique information, such as health, mana, etc, then it would still be very small, likely under 200 values. So when does it make sense to start using an embedding, as opposed to using the values directly? What's the underlying thought?

Answer 1

The advantages of embedding is converting categorical variables to numeric variables. Those numerical variables have semantic meaning. Thus any valid mathematical operation for numerical values has interpretable meaning.

One-hot encoding just tells the model that two entities are different. Embedding tells how different the entities are.