I am using word2vec to train a big corpus of textual information. While analyzing the scatter plot of words I found that it has not only grouped numbers together but also ordered them based on their value. I am able to understand that it groups together words sharing similar context but not able to understand how it got the values also correctly. Can someone please give an intuitive explanation of how this happens? I am attaching the image where I have zoomed in on the numbers.
Attempting an answer from the understanding I have of word2vec and deep learning through my personal experience and some live presentations of Yoshua Bengio and Yann Le Cun that I had the chance to attend.
As mentioned in the question, I think too that numbers are grouped together as they are have similar functions in the sentences where they are found.
For the reason of how they get ordered, as said in my comment: Intuitively I would say that it will be true for small numbers and less true for big numbers (the way 7-9-8 are ordered seems to go that way). When we speak we often say things like: "I ate 1 or 2 apples" or "I will buy 2 or 3 things" but rarely "I will buy 3 or 2 things". I think this is how it ends up ordering the numbers. Bigger numbers will appear less in a corpus so they will most probably not be ordered as precisely.
When looking at tutorials about word2vec, an example that is often given is that, if we see the relationship between 2 words as a vector, the final representation of the words by the model will give that the vector linking "man" to "woman" is approximately the same as the one linking "king" to "queen". Or that the vector linking "Paris" to "France" is approximately equal to the one linking "Rome" to "Italy". This even allows to do operations such as: France + Italy - Paris = Rome! The clear mechanics of how these relationships are so accurate (compared to other approaches) are, as of today, still obscur to the scientific community.
If these kind of operations works for countries and main cities, it seems logical that the vectors "3-2" and "2-1" point in the same direction, i.e. that 1-2-3 are ordered (given that, as said previously, consecutive numbers are more likely to be used together in a sentence). Of course, to achieve this, the numbers have to frequently appear in the training corpus.
I am very happy that someone showed this result with numbers because the way word2vec has been trained in most of the models that are available online did not includ digits. They were replaced by some symbols i.e., 12.34 becoming ##.##, and such observations cannot be made.
Any suggestion to improve this answer is welcome!