26
votes
Accepted
How do I load FastText pretrained model with Gensim?
Here's the link for the methods available for fasttext implementation in gensim fasttext.py
...
- 733
23
votes
How to initialize a new word2vec model with pre-trained model weights?
Thank Abhishek. I've figure it out! Here are my experiments.
1). we plot a easy example:
...
- 331
16
votes
How do I load FastText pretrained model with Gensim?
For .bin use: load_fasttext_format() (this typically contains full model with parameters, ngrams, etc).
For .vec use: ...
- 261
14
votes
Accepted
Doc2vec(gensim) - How can I infer unseen sentences’ label?
The title of this question is a separate question to its text so I will answer both separately (given that one leads into the other).
How can I infer unseen sentences:
...
- 288
12
votes
Word2Vec how to choose the embedding size parameter
I have checked four well-cited papers related to word embedding: 2013 Word2Vec, 2014 GloVe, 2018 BERT, and 2018 ELMo. Only GloVe has experimented on the embedding dimension for the analogy task (...
- 9,247
11
votes
Accepted
Word2Vec how to choose the embedding size parameter
You might find this paper might be the closest thing to what you are looking for if you don't want to treat it as a regular hyperparameter: Towards Lower Bounds on Number of Dimensions for Word ...
- 4,103
9
votes
How do I load FastText pretrained model with Gensim?
Update 04/2020
load_fasttext_format() is now deprecated, the updated way is to load the models is with ...
- 191
9
votes
Accepted
Difference between Gensim word2vec and keras Embedding layer
Yep, you're right! As you know, it's difficult for machine learning models to use natural language directly, so it helps to transform words into some meaningful numeric representation. This process ...
- 2,654
7
votes
Number of epochs in Gensim Word2Vec implementation
Increasing the number of epochs usually benefits the quality of the word representations. In experiments I have performed where the goal was to use the word embeddings as features for text ...
- 411
7
votes
Accepted
Why is averaging the vectors required in word2vec?
The reason to average the embedded vectors of the words in a paragraph or document is to obtain a single fixed-size vector that represents the whole text. Then, the document-level vector can be used ...
- 23.8k
6
votes
Accepted
In doc2vec, how to model correctly when many documents share the same label?
I've tried to explain the logic behind labels used in Document vectors in Doc2Vec - How to label the paragraphs (gensim)
To answer your questions.
1) when two documents share the same label, then ...
- 1,954
5
votes
Accepted
Antonym search for expanding search terms
That's an interesting problem.
This according to me is the most comprehensive way(if speed is not a problem. Or you can just pull all these words and create a dictionary/database of your own).
You ...
5
votes
How to initialize a new word2vec model with pre-trained model weights?
Let us look at a sample code:
...
- 2,316
5
votes
Number of epochs in Gensim Word2Vec implementation
I trained my w2v model on google news 300 for [2, 10, 100] epochs and the best one was on 10 epochs. After all that waiting, I was shocked that 100 epochs was bad.
...
- 150
5
votes
How to train an existing word2vec gensim model on new words?
Add this line before training with the new terms.
model.build_vocab([['potoatoes', 'and', 'farmers']], update=True)
After training,
...
5
votes
Word2Vec how to choose the embedding size parameter
Generally, the exact number of embedding dimensions does not affect task performance.
The number of dimensions can affect training time.
A common heuristic is to pick a power of 2 to speed up training ...
- 20.5k
5
votes
word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7
Let us try and understand how Word2Vector actually works before looking at distances:
There are 2 ways of generating vectors for a word :
Continuous bag of words
Skip grams
The following diagram ...
- 765
4
votes
Number of epochs in Gensim Word2Vec implementation
I looked here, and found that the default value changed from 1 to 5. Apparently the authors believe that more epochs will improve the results.
I cannot tell from experience, yet.
- 56
4
votes
Number of epochs in Gensim Word2Vec implementation
You can use a call back to output the loss at every epoch to help you decide how many to use:
...
4
votes
Accepted
Document Categorization Problem
Except for the OCR part, the right bundle would be pandas and sklearn.
You can check this ipython notebook which uses ...
- 156
4
votes
How do I load FastText pretrained model with Gensim?
I really wanted to use gensim, but ultimately found that using the native fasttext library worked out better for me. The following code you can copy/paste into ...
4
votes
Accepted
Which algorithm Doc2Vec uses?
Word2Vec is not a combination of two models, rather both are variants of word2vec. Similarly doc2vec has Distributed Memory(DM) model and Distributed Bag of words (DBOW) model. Based on the context ...
- 1,954
3
votes
Accepted
Learning character embeddings with GenSim
I believe that you misunderstood the word2vec concept. Basically for words, the feature vector for a word is learnt from the surrounding words.
...
- 1,954
3
votes
Accepted
Does traning the Word2Vec model multiple times affect `min_count` parameter?
The question has been answered in google groups by Gordon mohr.
Normally there's one read of the corpus to build the vocabulary (which includes initializing the model based on the learned vocabulary ...
- 1,954
3
votes
Antonym search for expanding search terms
Word2Vec can be used to find a word that relates to another word in the same way an example pair does. (fi: x is to happy, as bad is to good). You could use that to generate candidates of antonyms on ...
- 1,354
3
votes
Accepted
What are real world applications of Doc2Vec?
I will answer your second question first, doc2vec and word2vec both are primarily good representations of text data that capture the semantics of words and documents. So whenever you are working with ...
- 1,848
3
votes
Accepted
When to use different Word2Vec training approaches?
The advantage of using pre-trained vectors is being able to inject knowledge from a larger corpus than you might have access to: word2vec has a vocabulary of 3 million words and phrases trained on the ...
- 1,668
3
votes
Sub topics with Latent Dirichlet Allocation
I actually do not think your method is a good way to find subtopics. Consider a document X with a distribution of topics z. X is made up of a mixed model distribution of topic Z. If you just give a ...
- 2,400
3
votes
Accepted
Metrics for unsupervised doc2vec model
Evaluation is based on the task, not the type of model used for it. In the tutorial that you link the task would be simple document similarity. Afaik a more common variant is the information retrieval ...
- 25k
3
votes
Accepted
How to test the quality of a word embedding?
One way to test your embedding is see how often your model agrees with the common consensus of how other embeddings complete word analogies. A collection of established word embedding analogies are ...
- 20.5k
Only top scored, non community-wiki answers of a minimum length are eligible