I'm trying to obtain the word2vec representation of few words using gensim.

At present, this is the model that I have:

model = Word2Vec(desc, size=1, seed=7, window=1, iter=1000)

And the current output is

Ansible automate 0.37919244170188904
Docker container -0.05218552052974701
Ansible automation 0.39010459184646606
Automate Ansible 0.1009570062160492
Contain Docker 0.41532352566719055
Dock contain -0.38013115525245667
Dock container -0.17561538517475128

I'm wondering why Contain Docker is closer to Ansible automate and Ansible automation than it is from the other terms which contain the word Docker, Container etc.

How can I train the word2vec model to get this sorted out?


First of all, consider this: Word2Vec generates this representation by looking at the context (neighboring words, here with a window of 1) of each word or n-gram (e.g. pair of words such as Contain Docker).

Your are using a uni-dimensional representation of those pairs of words. Even in a particular training text such as yours which seems to be revolving around a unique subject (i.e. docker), a unique value as representation means a unique dimension in the "semantic space". This yields really poor representation power, and Word2Vec models are usually trained with more than 300 hundreds dimension to fully capture the semantic of words. And usually, more is better (but not always).

I hence suggest that you increase the size of your vectors (try different values, such as 100 and 200).

To see if the new representation makes sense, I suggest to use Multidimensional Scaling, which enables to project N-dimensional data on a plan. See for instance the answers to this question

What if you still want to use a uni-dimensional encoding ?

For a training text about emotions, it could mean a positive vs. negative dimension (example representations: happy = 0.85, sad = -0.94). In your case, I have no idea and suggest to plot a selection of words along this axis, to see if there are any obvious pattern for having high value or low value.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.