Questions tagged [word-embeddings]
Word embedding is the collective name for a set of language modeling and feature learning techniques in NLP where words are mapped to vectors of real numbers in a low dimensional space, relative to the vocabulary size.
507
questions
8
votes
2
answers
2k
views
How does word2vec handle the input word being in the context?
If word2vec encounters the same word multiple times in the same window, what occurs? Obviously it is meaningless to decrease the distance between the vectors for the input word and the target word. ...
- 3,077
8
votes
2
answers
6k
views
How word2vec can handle unseen / new words to bypass this for new classifications?
In simple terms, if my classification is based on word2vec as features, what I am supposed to do, if a new word comes, which does not have a word2vec?
I am trying to used word2vec or word vectors for ...
- 81
2
votes
2
answers
3k
views
Program to fine-tune pre-trained word embeddings on my data set
I am looking for a program that would allow me to fine-tune pre-trained word embeddings on my data set. Ideally, open source and working on Linux or Windows.
- 5,630
10
votes
2
answers
12k
views
Reducing the dimensionality of word embeddings
I trained word embeddings with 300 dimensions. Now, I would like to have word embeddings with 50 dimensions: is it better to retrain the word embeddings with 50 dimensions, or can I use some ...
- 5,630
10
votes
1
answer
5k
views
How much training data does Word2Vec need?
I'd like to compare the difference among the same word mentioned in different sources. That is, how authors differ in their usage of ill-defined words, such as "democracy".
A brief plan was
Take the ...
- 711
12
votes
2
answers
10k
views
Shall I use the Euclidean Distance or the Cosine Similarity to compute the semantic similarity of two words?
I want to compute the semantic similarity of two words using their vector representations (obtained using e.g. word2vec, GloVe, etc.). Shall I use the Euclidean Distance or the Cosine Similarity?
The ...
- 5,630
3
votes
1
answer
5k
views
Gradient Descent Step for word2vec negative sampling
For word2vec with negative sampling, the cost function for a single word is the following according to word2vec:
$$
E = - log(\sigma(v_{w_{O}}^{'}.u_{w_{I}})) - \sum_{k=1}^K log(\sigma(-v_{w_{k}}^{'}....
- 63