I calculate vector similarities like this:
nlp = spacy.load('en_trf_xlnetbasecased_lg') a = nlp("car").vector b = nlp("plant").vector dot(a, b)/(norm(a)*norm(b)) 0.966813
Why are the vector similarities so high for unrelated words for the embedding? This is not the only pair for which they are abnormally high. I also had a similar experience with fastText, so I am wondering, am I misunderstanding something?
Also I am able to get vectors for non-words like "asdfasfdasfd" or "zzz123Y!/§zzzZz", and they differ from each other. How is this possible?