Skip to main content
27 votes
Accepted

How do I load FastText pretrained model with Gensim?

Here's the link for the methods available for fasttext implementation in gensim fasttext.py ...
Sabbiu Shah's user avatar
23 votes

How to initialize a new word2vec model with pre-trained model weights?

Thank Abhishek. I've figure it out! Here are my experiments. 1). we plot a easy example: ...
Shixiang Wan's user avatar
16 votes

How do I load FastText pretrained model with Gensim?

For .bin use: load_fasttext_format() (this typically contains full model with parameters, ngrams, etc). For .vec use: ...
Akash Kandpal's user avatar
14 votes
Accepted

Doc2vec(gensim) - How can I infer unseen sentences’ label?

The title of this question is a separate question to its text so I will answer both separately (given that one leads into the other). How can I infer unseen sentences: ...
Francisco Vargas's user avatar
13 votes

Word2Vec how to choose the embedding size parameter

I have checked four well-cited papers related to word embedding: 2013 Word2Vec, 2014 GloVe, 2018 BERT, and 2018 ELMo. Only GloVe has experimented on the embedding dimension for the analogy task (...
Esmailian's user avatar
  • 9,342
12 votes
Accepted

Word2Vec how to choose the embedding size parameter

You might find this paper might be the closest thing to what you are looking for if you don't want to treat it as a regular hyperparameter: Towards Lower Bounds on Number of Dimensions for Word ...
Simon Larsson's user avatar
10 votes

How do I load FastText pretrained model with Gensim?

Update 04/2020 load_fasttext_format() is now deprecated, the updated way is to load the models is with ...
jcaliz's user avatar
  • 201
9 votes
Accepted

Difference between Gensim word2vec and keras Embedding layer

Yep, you're right! As you know, it's difficult for machine learning models to use natural language directly, so it helps to transform words into some meaningful numeric representation. This process ...
zachdj's user avatar
  • 2,734
7 votes

Number of epochs in Gensim Word2Vec implementation

Increasing the number of epochs usually benefits the quality of the word representations. In experiments I have performed where the goal was to use the word embeddings as features for text ...
geompalik's user avatar
  • 411
7 votes
Accepted

Why is averaging the vectors required in word2vec?

The reason to average the embedded vectors of the words in a paragraph or document is to obtain a single fixed-size vector that represents the whole text. Then, the document-level vector can be used ...
noe's user avatar
  • 27k
6 votes
Accepted

In doc2vec, how to model correctly when many documents share the same label?

I've tried to explain the logic behind labels used in Document vectors in Doc2Vec - How to label the paragraphs (gensim) To answer your questions. 1) when two documents share the same label, then ...
chmodsss's user avatar
  • 1,964
6 votes

How to train an existing word2vec gensim model on new words?

Add this line before training with the new terms. model.build_vocab([['potoatoes', 'and', 'farmers']], update=True) After training, ...
Thirupathi Thangavel's user avatar
6 votes

Word2Vec how to choose the embedding size parameter

Generally, the exact number of embedding dimensions does not affect task performance. The number of dimensions can affect training time. A common heuristic is to pick a power of 2 to speed up training ...
Brian Spiering's user avatar
5 votes

How do I load FastText pretrained model with Gensim?

I really wanted to use gensim, but ultimately found that using the native fasttext library worked out better for me. The following code you can copy/paste into ...
information_interchange's user avatar
5 votes
Accepted

Antonym search for expanding search terms

That's an interesting problem. This according to me is the most comprehensive way(if speed is not a problem. Or you can just pull all these words and create a dictionary/database of your own). You ...
Rahul Reddy Vemireddy's user avatar
5 votes

How to initialize a new word2vec model with pre-trained model weights?

Let us look at a sample code: ...
Hima Varsha's user avatar
  • 2,346
5 votes

Number of epochs in Gensim Word2Vec implementation

I trained my w2v model on google news 300 for [2, 10, 100] epochs and the best one was on 10 epochs. After all that waiting, I was shocked that 100 epochs was bad. ...
MasterOne Piece's user avatar
5 votes

word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7

Let us try and understand how Word2Vector actually works before looking at distances: There are 2 ways of generating vectors for a word : Continuous bag of words Skip grams The following diagram ...
Nischal Hp's user avatar
4 votes

Number of epochs in Gensim Word2Vec implementation

I looked here, and found that the default value changed from 1 to 5. Apparently the authors believe that more epochs will improve the results. I cannot tell from experience, yet.
H.M. Prins's user avatar
4 votes

Number of epochs in Gensim Word2Vec implementation

You can use a call back to output the loss at every epoch to help you decide how many to use: ...
Damian Satterthwaite-Phillips's user avatar
4 votes
Accepted

Which algorithm Doc2Vec uses?

Word2Vec is not a combination of two models, rather both are variants of word2vec. Similarly doc2vec has Distributed Memory(DM) model and Distributed Bag of words (DBOW) model. Based on the context ...
chmodsss's user avatar
  • 1,964
3 votes
Accepted

Learning character embeddings with GenSim

I believe that you misunderstood the word2vec concept. Basically for words, the feature vector for a word is learnt from the surrounding words. ...
chmodsss's user avatar
  • 1,964
3 votes

Antonym search for expanding search terms

Word2Vec can be used to find a word that relates to another word in the same way an example pair does. (fi: x is to happy, as bad is to good). You could use that to generate candidates of antonyms on ...
S van Balen's user avatar
  • 1,364
3 votes
Accepted

What are real world applications of Doc2Vec?

I will answer your second question first, doc2vec and word2vec both are primarily good representations of text data that capture the semantics of words and documents. So whenever you are working with ...
Himanshu Rai's user avatar
  • 1,848
3 votes
Accepted

When to use different Word2Vec training approaches?

The advantage of using pre-trained vectors is being able to inject knowledge from a larger corpus than you might have access to: word2vec has a vocabulary of 3 million words and phrases trained on the ...
redhqs's user avatar
  • 1,678
3 votes

Sub topics with Latent Dirichlet Allocation

I actually do not think your method is a good way to find subtopics. Consider a document X with a distribution of topics z. X is made up of a mixed model distribution of topic Z. If you just give a ...
Tophat's user avatar
  • 2,430
3 votes
Accepted

Metrics for unsupervised doc2vec model

Evaluation is based on the task, not the type of model used for it. In the tutorial that you link the task would be simple document similarity. Afaik a more common variant is the information retrieval ...
Erwan's user avatar
  • 25.5k
3 votes
Accepted

How to test the quality of a word embedding?

One way to test your embedding is see how often your model agrees with the common consensus of how other embeddings complete word analogies. A collection of established word embedding analogies are ...
Brian Spiering's user avatar
2 votes

How to initialize a new word2vec model with pre-trained model weights?

If you are looking for a pre-trained net for word-embeddings, I would suggest GloVe. The following blog from Keras is very informative of how to implement this. It also has a link to the pre-trained ...
Samuel Sherman's user avatar
2 votes

Antonym search for expanding search terms

You can start with small lists of antonyms like this one. Maybe you can get a comprehensive list that will take care of most of the cases you are interested in. For now, let assume that you don't have ...
DaL's user avatar
  • 2,643

Only top scored, non community-wiki answers of a minimum length are eligible