@adosar I see at least two potential issues with this. Using scalars, for example integers, you can get very large numbers, and neural networks usually don't work very well with large numbers. So you may try to normalize them to the range [0,1], but then if you add a new word, the whole representation changes after normalization. Additionally, you lose some properties of the representation using vectors, which you can use to get some interesting stuff like king - man + woman = queen