Questions tagged [doc2vec]

The tag has no usage guidance.

13 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
2 answers
752 views

Gensim doc2vec error: KeyError: "word 'senseless' not in vocabulary"

I am new to machine learning and tried doc2vec on quora duplicate dataset. new_dfx has columns 'question1' and 'question2' which has preprocessed questions in each row. Following is the tagged ...
Ankit Rohilla's user avatar
2 votes
0 answers
59 views

Preprocessing for Document Similarity Using Doc2Vec

I'm trying to determine document similarity using Doc2Vec on a large series of legal opinions, which can contain some highly jargonistic language and phrases (e.g. en banc, de novo, etc.). I'm ...
user118648's user avatar
2 votes
0 answers
133 views

What is the meaning of, or explanation for, having multiple tags in a Doc2Vec model's TaggedDocuments?

I've tried reading the other answers on this topic but I'm unsure if I understand completely. For my dataset, I have a series of tagged documents, "good" or "bad." Each document ...
Jayke's user avatar
  • 21
2 votes
1 answer
316 views

How to implement LSTM using Doc2Vec vectors to get representation?

Hi all. I'm a newbie in ML. I read and found a paper about A Multi-Level Plagiarism Detection System Based on Deep Learning Algorithms and want to implement this model . But I can't find more about ...
Omasaka Opacha Revok's user avatar
1 vote
1 answer
295 views

Treating Word Embeddings as Multivariate Gaussian Random Variables

I want to specify some probabilistic clustering model (such as a mixture model or lda) over words, and instead of using the traditional method of representing words as an indicator vector , I want to ...
ricardo's user avatar
  • 23
1 vote
1 answer
36 views

Is there a way to train Doc2Vec on a corpus of docs and be able to take a novel doc and see how similar it is to the trained corpus?

I have a project idea, where I train a bunch of documents on Doc2Vec and then take a novel, input doc, and ideally be able to be told how similar it is to the docs supplied for training as a whole or ...
sangstar's user avatar
  • 133
1 vote
1 answer
1k views

Embedding from Transformer-based model from paragraph or documnet (like Doc2Vec)

I have a set of data that contains the different lengths of sequences. On average the sequence length is 600. The dataset is like this: ...
Bloodstone Programmer's user avatar
1 vote
0 answers
63 views

doc2vec - paragraph or article as document

I'm trying to train a doc2vec model on the German wiki corpus. While looking for the best practice I've found different possibilities on how to create the training data. Should I split every Wikipedia ...
jonas's user avatar
  • 143
1 vote
1 answer
126 views

Topic alignment / topic modelling

What is the most efficient method for detecting whether the article is mostly about a specific topic, but without lots of data for training? My task is to determine how much a document is e.g. about ...
piernik's user avatar
  • 51
1 vote
0 answers
188 views

T-SNE good clustering but SVM classification poor

I am trying to classify in 4 different classes, paragraph embedding vector computed with doc2vec using an non-linear svm over them. When I visualize the embeddings using tensorboard t-sne I can see ...
Luca Massarelli's user avatar
1 vote
0 answers
20 views

Can feature representation acquired by a same model but trained on different corpus be used on the same classification model?

For example, if I wanna do document classification with doc2vec embeddings, first I train the training set to get doc2vec embeddings, and fit the embeddings to a classification model; later on when I ...
Watercake's user avatar
0 votes
1 answer
57 views

Why do we want to maximize the average log probability in neural language models?

I am currently trying to understand the Paragraph Vector framework by reading the paper "Distributed Representation of Sentences and Documents" by Quoc Le and Thomas Mikolov but I have ...
LeeKed's user avatar
  • 1
0 votes
1 answer
62 views

Vector representation of documents for text classification

I'm looking for proper method of document embeddings. I know that doc2vec will give me the vector representations for given corpus, but how do I embed new documents? I need to train neural network ...
Mikołaj Wróblewski's user avatar