Questions tagged [word2vec]

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

Filter by
Sorted by
Tagged with
1
vote
0answers
20 views

Literature on selecting specific dimensions in a word embedding vector

I am aware that the different dimensions in the word embedding represents different information and algebraic operations can be performed between two embeddings for example. Can anyone point me to ...
1
vote
1answer
13 views

How to identify text similarity based on training data?

I have a set of documents (1 to 11) for which the labeling is done. Lets Assume: ...
0
votes
1answer
65 views

Embedding of list of objects

I have a dataset where each sample is a list of ordered items, lets say grocery list , and a label from 6 categories . each list can have up to 120 items but the mean items is 12 items in a list. i ...
1
vote
2answers
26 views

How can I encode a 'Name' so that similar names are represented by vectors close in n-dimensional plane?

I want to encode names of people for similarity comparison between them such that a name like 'Sarah' is closer when represented in vector to a name like 'Sarah connor', something very similar to what ...
1
vote
0answers
24 views

Extracting vectors of FastText own model to use it on a NN

I have trained my own model of fasttext using the pretrained model of English available on their website with the next code: ...
0
votes
0answers
12 views

Inserting input representation at each step of LSTM

In want to train a neural net to generate lyrics based on a provided melody. For that, I have to implement a recurrent neural net (LSTM or GRU) to carry out the task. During each step of the training ...
1
vote
0answers
51 views

How to group various similar search keywords and find top 100 keywords from big dataset

I have search keywords in one of my database table. These are the keywords searched by users on a website. My requirement is to find the top 100 search keywords after consolidating various similar ...
0
votes
2answers
42 views

Word2Vec with CNN

I am trying to classify documents using CNN (convolutional neural network) with Word2Vec embeddings. However to do this, it requires me to trim all texts to the same length. I just pad all the ...
1
vote
0answers
18 views

Topic alignment / topic modelling

What is the most efficient method for detecting whether the article is mostly about a specific topic, but without lots of data for training? My task is to determine how much a document is e.g. about ...
0
votes
0answers
13 views

How effective would this pseudo-LDA2Vec implementation be?

For my site I'm working on a chat recommender that would recommend chats to users. Each chat has a title and description and my corpus is composed of many of these title and description documents. I ...
0
votes
0answers
23 views

T-SNE good clustering but SVM classification poor

I am trying to classify in 4 different classes, paragraph embedding vector computed with doc2vec using an non-linear svm over them. When I visualize the embeddings using tensorboard t-sne I can see ...
0
votes
1answer
26 views

Word2Vec Implementation

In word2vec why is the implementation of likelihood function multiplication of probabilities of finding a neighbouring word given a word? I didnt get why the probabilities should be multiplied.Is ...
1
vote
0answers
21 views

Semi-Supervised Learning using NLP

I am working on a drug reaction problem in which I need to extract tweets and label the tweets (binary-reaction due to drug or not). But since I don't have domain knowledge, and clustering would also ...
1
vote
0answers
13 views

Using NLP in already analysed text,

I have serveral text files. These files has been analysed through some analytical tool and provided main features There each feature extracted has one repetition I know to use predictive modeling ...
2
votes
1answer
154 views

Predicting the missing word using fasttext pretrained word embedding models (CBOW vs skipgram)

I am trying to implement a simple word prediction algorithm for filling a gap in a sentence by choosing from several options: Driving a ---- is not fun in London streets. Apple Car Book King With ...
3
votes
1answer
33 views

What are machine learning/deep learning models for generating contextually related words and synonyms?

I have a task to work on models for finding synonyms and contextually related words. For example, if I enter: 'car' it should generate -> 'vehicle' 'sun' and 'sea' could generate 'beach', or some ...
0
votes
0answers
38 views

why does adding an LDA document vector with a word2vec word vector work well in LDA2vec?

In LDA the document weight vector represents the "weights" of each topic in the document. I think it's also valid to say, each row in the document vector corresponds to a word in the document, the ...
0
votes
1answer
126 views

Why does accuracy improve when vocabulary size of word embedding model is reduced?

I am quite new to the topic of word embedding using word2vec and models such as skip-gram. Our teacher introduced us to this TensorFlow code on word2vec which he ran on Google Colab notebook. He ...
0
votes
1answer
491 views

How to preprocess data for Word2Vec?

I have text data which is crawled from websites. I am preprocessing data to train Word2Vec model. Should I remove stopwords and do lemmatization? How to preprocess data for Word2Vec?
0
votes
0answers
8 views

Adding feed forward layer for word2vec/doc2vec in hidden layers

Word2vec and paragraph2vec(doc2vec) both adopt very simple strucutre -- input layer, hidden layer which concatenate or averaging the input layer, and softmax output layer. If one add more hidden ...
1
vote
0answers
17 views

Are vectors generated by doc2vec and similar models uniformly distributed?

I have read that vectors in a word2vec model are very much not uniformly distributed and are thought to follow Zipf's law; is this the same for the associated models like paragraph2vec, doc2vec, etc? ...
1
vote
1answer
865 views

Word2Vec and Tf-idf how to combine them

I'm currently working in text mining ptoject I'd like to know once I'm on vectorisation. With method is better. Is it Word2Vec or ...
2
votes
1answer
41 views

Word Embedding or Hash?

In my dataset I have a 'text' column and a 'followers' column containing lists of follower IDs, i.e. '1093777852477116417, 936194589043683328,...'. Some of the 'followers' values contain thousands of ...
1
vote
1answer
24 views

How does backpropagation work with averaging layers?

I'm studying Word2Vec algorithm, and so far i understood that, in the case of input context bigger than 1 (so multiple words) we have our hidden layer that performs averaging between the inputs (as ...
3
votes
1answer
91 views

Can 2 different OOV words get the same vector in FastText?

Since FastText sums up the vectors(order is not considered) of an OOV word's subwords, is it possible for two different OOV words to get the same vector ? If so, then can you give an example?
1
vote
1answer
62 views

What are x variable and y variable in word2vec model if it is supervised learning

What are x variable and y variable in word2vec model if it is supervised learning. In both the flavours- CBOW and skip-gram model. Though some blogs have explained it as unsupervised learning. ...
0
votes
0answers
57 views

Nan loss on training Keras regression model

I have defined the following model in Keras wherein I am trying to apply regression on some data. My input dimension is of size (300, 250) where each of the 300 values represents a cluster center and ...
1
vote
0answers
16 views

Can feature representation acquired by a same model but trained on different corpus be used on the same classification model?

For example, if I wanna do document classification with doc2vec embeddings, first I train the training set to get doc2vec embeddings, and fit the embeddings to a classification model; later on when I ...
1
vote
1answer
111 views

Do repeated sentences impact Word2Vec?

I'm working with domain-oriented documents in order to obtain synonyms using Word2Vec. These documents are usually templates, so sentences are repeated a lot. 1k of the unique sentences represent 83%...
1
vote
1answer
16 views

help finding research discussion on HTS classification

My question is about the theory of this problem, and not necessarily syntax. I'm wondering if anyone here has experience with automating HTS (Harmonized Tax Schedule) classifications, specifically ...
1
vote
0answers
13 views

FastText and CharEmbedding

i have got a question. i don't understand how to develop my DLModel. I'm working on DGA Detection with this type of dataset: ...
3
votes
1answer
962 views

Difference between Gensim word2vec and keras Embedding layer

I used the gensim word2vec package and Keras Embedding layer for various different projects. Then I realize they seem to do the ...
1
vote
2answers
275 views

How to find similar phrases

I have the following problem: I have created a customized Dictionary for getting used in some NLP tasks. I want to enhance my dictionary by finding phrases similar to the phrases I have in my ...
0
votes
0answers
43 views

Intuition for inference of doc2vec models, on document parts

I am trying to understand how doc2vec models perform during inference on documents when we split them in various ways. Example document: ...
0
votes
0answers
639 views

Interpreting Gensim Word2Vec Training Loss

I am using Gensim to build a Word2Vec model and identify the convergence of training loss, so that I can figure out the optimal number of iterations. For understanding this since gensim's ...
1
vote
2answers
78 views

how to use word embedding to do document classification etc?

I just start learning NLP technology, such as GPT, Bert, XLnet, word2vec, Glove etc. I try my best to read papers and check source code. But I still cannot understand very well. When we use word2vec ...
1
vote
0answers
12 views

does the model update for every word encountered in word2vec?

In skipgram negative sampling according to the author's implementation, does the model update with every word? https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-...
0
votes
1answer
17 views

Models after word2vec outputs

I am originally using a bag of word (2-gram) model to approach a classification problem. The one hot encoding of the 2-gram output was sent to a logistic regression or neural network to build a ...
1
vote
1answer
64 views

can we make a word2vec NN of more than 3 layers using tensorflow?

To the best of my understanding , word2vec crated using gensim is of 3 layers only. I was wondering can we customize word2vec NN and create word2vec NN of more than 3 layers to experiment with it ...
0
votes
1answer
39 views

Why are bigger embedding vectors not necessarily better?

I'm wondering why increasing the dimension of a word dimension vector in NLP doesn't necessarily lead to a better result. For instance, on examples I run, I see sometimes that using a pre-trained 100d ...
1
vote
2answers
257 views

Where can I find dataset for word analogy task?

In the paper of Word2Vec by Thomas Mikolov and others, there is a accuracy report on the full Semantic-Syntactic data set. Where I can find this dataset or a related dataset for word analogy task? ...
2
votes
1answer
28 views

Unable to learn weights of a Word2Vec model

I was going to implement a word embedding model - namely Word2Vec - by following this TensorFlow tutorial and adapting the code a little bit. Unfortunately, though, my model won't learn anything. I've ...
1
vote
1answer
27 views

How to maintain CBOW dataset dimension and fit it in Neural Network?

I am new to neural network. I'm trying to train word embeddings without using word2vec package. Using titles from reddit worldnews dataset I'm have done some CBOW representation. For window size ...
0
votes
0answers
6 views

Training data for doc2vec models, general vs specific

I have quite a general question about doc2vec models. Let's say I have a specific NLP task whose goal is to understand the similarity between two sports news articles. Now I have the option to train ...
1
vote
1answer
238 views

Metrics for unsupervised doc2vec model

I have just built a simple doc2vec model using the gensim library, pretty much followed the tutorial located here. The methods provided for checking the quality of the model are very manual and ...
1
vote
2answers
316 views

Is it meaningful to use word2vec for non-string inputs like time series analysis?

I am working on a project that detects anomalies in a time series. I wonder if I can use word2vec for anomaly detection for non-string inputs like exchange rates?
1
vote
0answers
13 views

How can one determine that Word2Vec (CBOW method) embeddings are related to each other?

I read some fascinating stuff about the potential for using the Word2Vec algorithm to speed up the pace of scientific discovery here https://www.researchgate.net/publication/...
1
vote
0answers
587 views

How to effectively tune the hyper-parameters of Gensim Doc2Vec to achieve maximum accuracy in Document Similarity problem?

I have around 20k documents with 60 - 150 words. Out of these 20K documents, there are 400 documents for which the similar document are known. These 400 documents serve as my test data. At present I ...
0
votes
0answers
17 views

Outputs of a Doc2Vec model

I have trained a Doc2Vec model and I am interested in some of the model outputs. I am specifically interested to see if it is posible to obtain the Word embeddings from the Doc2Vec model, or obtain a ...
1
vote
1answer
116 views

Word2Vec - document similarity

Lets say I have text data for different documents from 2005 - 2015. I want to compare the similarity between $t$ and $t-1$ documents. So I take the document at 2006 and compare it with the document at ...

1
2 3 4 5