Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

word2vec is a two layer neural network to process text. It takes words as an input and outputs a vector correspondingly. It uses a combination of Continuous Bag of Word and skipgram model implementation.

1
vote
1answer
8 views

How to count number of word embeddings in Gensim Word2Vec model

I am trying to create a Word2Vec model of the the Pub Med Central corpus using the Gensim library and want to limit the total number of word embeddings to around 1 billion. I have searched high and ...
1
vote
0answers
9 views

Learning word embeddings using RNN

The common way of learning word embeddings is based on BOW, and Skip-gram models. Is it possible to train a RNN-based architecture like GRU or LSTM with random sentences from a large corpus to learn ...
0
votes
0answers
11 views

Adding and Normalizing extra features to Word2Vec representation

My problem is kind of similar to this question I am currently using a word2vec 100 features representation of my words. However, I want to add more features to have more similarity between synonyms ...
2
votes
1answer
80 views

Semi Supervised Learning without label propagation

I am trying to cluster some words by affinity. Using Word2Vec I obtained vector representation of every word that I can cluster with a normal unsupervised method. Of these words, though, I know the ...
0
votes
1answer
46 views

Text Similarities: which nlp methods to use?

I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text. Question 1: Which NLP method should I start with? I have tried ...
0
votes
1answer
26 views

how to update the pre-trained word2vec model with new train data using genism

Hi I have used the genism to load the Spanish fasttext word2vec model with following code: ...
1
vote
1answer
18 views

Why does spark.ml.feautures.Word2Vec vectorize sentences instead of single words?

In the process of understanding how Word2Vec in Spark differs from gensim one, I got very confused by the example presented in the Spark docs (reference link: https://spark.apache.org/docs/2.2.0/ml-...
-1
votes
0answers
23 views

Machine Learning based software vulnerability scoring

A challenge in scoring vulnerabilities detected in web applications and REST-based applications is outright automation of the severity scoring process. While software packages are scored using the ...
0
votes
0answers
14 views

Extract/detect IDs (like flight booking ids) from text

I'm looking to extract ids from the body of an email. The ids are similar to flight booking IDs. For example, in an email, I would like to obtain the booking reference (something like MNFF3RGC or MNF-...
0
votes
1answer
28 views

Why would you use word embeddings to find similar words?

One of the applications of word embeddings (such as GloVe) is finding words of similar meaning. I just had a look at some embeddings produced by glove on large datasets and I found that the nearest ...
0
votes
1answer
21 views

Doc2Vec network architecture

I have been familiarizing myself with Word2Vec and Doc2Vec. After reading multiple papers including the the ones by T Mikolov (the creator of Doc2Vec), I am not clear on how does the neural network ...
0
votes
0answers
18 views

Does it make sense to add word embeddings as additional features for LSTM model?

I have an LSTM model. This model takes as input tokens. Those tokens represent XML markups extracted from some XML files. My model is working fine. However, I want to optimize it by adding word ...
0
votes
1answer
27 views

what is the reason behind the bad outputs gained by RNN, LSTM when using GloVe pretrained model in text classification?

the problem is with the results gained for accuracy and f1 afer training our model via pretrained models such as GloVe. when I apply CNN as a classifier, the result are good as follows: ...
1
vote
0answers
19 views

Training of word weights in Word Embedding and Word2Vec

I want to know how are the word weights updated for the embedding layer in Keras and for Word2Vec. Like for the normal model.add(Embedding(..)) and ...
0
votes
0answers
60 views

is Glove better for word similarity Skip-gram/CBOW?

While looking at the slides for lecture 2 of CS224d: Deep Learning for Natural Language Processing: Link to slides It is said in slide number 31, that count based methods (ex: LSA) for creating word ...
0
votes
1answer
39 views

how to input the data set in to a word2vec by keras?

I am new in using word2vec model, as a result, I do not know how I can prepare my dataset as an input for word2vec? I have searched a lot but the datasets in tutorials were in CSV format or just one ...
1
vote
1answer
31 views

Would keeping all punctuation make any sense in word2vec?

I am trying to learn how word2vec works to get to more complicated stuff like LSTMs. Because I will use the same training data (so with the same vocabulary) and I want to predict punctuation too, I ...
0
votes
1answer
58 views

what is the reason behainde the Error :UnicodeDecodeError:'utf-8' codec can't decode byte 0x94 in position 19: invalid start byte

I am trying to apply open() function in keras to use Google news-vectors-negative300.bin which is a pre-trained file via word2vec such as GloVe, but after downloading GloVe it contains 4 files with ...
0
votes
0answers
36 views

Case-Sensitive Word Embeddings for French

Are there any pre-trained case-sensitive word embeddings for French? The only word embeddings for French I have found is FastText and it is not case sensitive. I am currently working on problems ...
0
votes
0answers
17 views

Improving clustering by incrementing density and number of elements in clusters?

Question I would like to know if, in general, clustering tecniques work better if the clusters have more elements related to them. I know that adding noise is a risk and would result in creating ...
1
vote
0answers
12 views

Are there any paper for a closed domain conversational agent

i was trying to find a closed domain conversational agent/chatbot paper in Question and answering so not long conversation, and i don't think i see any. All the paper i can find are related to an ...
1
vote
2answers
31 views

Clustering with arrays / vectors as features?

I am trying to cluster a large set of documents of which I have a DOC2VEC representation. But I want to cluster them with more features, thus resulting in having both a vector (...
1
vote
0answers
18 views

How is determined the context's dimension in Doc2Vec?

I would like to know how is determined the dimension of the context in Gensim Doc2Vec.
1
vote
0answers
45 views

Automatic Semantic Clustering and Tagging of sentences using NLP

NLP Analysis for keyword clustering I have a set of keywords for search engines and I would like to create a python script to classify and tag them under unknown categories. To make it clear I ...
2
votes
0answers
89 views

What is the minimum number of times a word needs to appear in word2vec training corpus for quality results?

When training a word2vec model with, eg, gensim, you can specify the minimum times a word needs to be seen (with the parameter min_count). The default value for this seems to be 5. Are there any ...
0
votes
1answer
64 views

What is the proper train data format in LSTM?

I want to train a model to detect wrong word using in sentence. I have 1 million sentences(word base or char base) with different length. Each position(word or char) has a label to indicate it is ...
4
votes
2answers
96 views

Automatic code checking

I have some experience in machine learning, mainly clustering and classifiers. However, I am somewhat of a newbie when it comes to NLP. That said I am aware of all the various issues and ...
4
votes
1answer
162 views

Accuracy and loss don't change in CNN. Is it over-fitting?

My task is to perform classify news articles as Interesting [1] or Uninteresting [0]. My training set has 4053 articles out of which 179 are Interesting. The validation set has 664 articles out of ...
3
votes
0answers
16 views

In Word2Vec, how to the vector values translate back to related words?

Apologies for the newbie question: I've just downloadd the GoogleNews word2vec bin file and used convertvec to convert it to a text file to look at the vector values. I see they are values between -1 ...
2
votes
2answers
132 views

Image Embeddings - Negative Sampling and Imbalanced Class Issues

I am using the negative sampling approach used in Word2Vec to train some image embeddings. From what I have read, for every positive example, we are creating a number of negative examples. Question: ...
1
vote
0answers
26 views

Question about CBOW prediction?

I have a question about CBOW prediction. Suppose my job is to use 3 surrounding words w(t-3), w(t-2), w(t-1)as input to predict one target word w(t). Once the model is trained and I want to predict a ...
4
votes
1answer
41 views

Skipgram - multiple formulations?

I've been reading about the Skipgram model and I have found what I interpreted as multiple definitions. 1 - Taking a look at this blog post and Andrew Ng's Deep Learning Specialization, I understood ...
3
votes
1answer
58 views

what actually word embedding dimensions values represent?

I am learning word2vec and word embedding , I have downloaded GloVe pre-trained word embedding (shape 40,000 x 50) and using this function to extract information from that: ...
0
votes
0answers
39 views

Pre-trained Word embedding model for conversational vocabulary?

I am currently using Google’s pre-trained Word2Vec model for word sentiment analysis, however, since the model is trained on news articles I found that it's not that effective on conversational texts. ...
0
votes
2answers
227 views

How to alter word2vec wikipedia model for n-grams?

I have a very little data, so my word2vec model does not perform well. My intention is to identify words similar to technical terms such as 'support vector machine', 'machine learning', 'artificial ...
1
vote
1answer
81 views

Confusion about Keras' skipgram and sampling table utilities

I'm fairly new to ML, so as a learning exercise to get familiar with Keras I'm trying to learn some word2vec style embeddings from a dataset. I'm confused about the behavior of the skipgram utility, ...
0
votes
0answers
29 views

Is it possible to have variable window size for Continuous Bag of Words method of training word embeddings?

All the literature I've seen so far in the CBOW model uses a fixed window size, ie window size of 2. Is it possible to have a variable window size? For example, one set will have 8 words for input ...
0
votes
0answers
94 views

How do I return Doc2Vec vectors of a corpus after training it using a pre-trained model?

I am trying to implement Doc2Vec model to convert a corpus into vectors using a pre-trained model (GoogleNews-vectors-negative300.bin). I want to return the ...
6
votes
1answer
265 views

Why do we need 2 matrices for word2vec or GloVe

Word2vec and GloVe are the two most known words embedding methods. Many works pointed that these two models are actually very close to each other and that under some assumptions, they perform a matrix ...
0
votes
1answer
22 views

Word2Vec benefit in LSTM

if Word2Vec is nothing but a transformation of one-hot into a dense vector, why can't I just feed one-hot into LSTM (or for that matter sacrifice first dense layer, in any network that will end up ...
0
votes
0answers
440 views

How to use word embedding vectors along with other features in a machine learning model?

Love this forum for all the help. Thanks for taking a look at my question. For my ML problem, I am tyring to use word embeddings(word2vec,cbag + tfidf) along with other dense features (example ...
1
vote
2answers
140 views

use hidden layer of word2vec instead of 'one-hot', to reduce number of weights in other nets?

I've been reading about word2vec and it's ability to encode words into vector representations. Coordinates (probabilities) of these words are clustered together with their usual context-neighbor words....
-1
votes
1answer
300 views

Bag of words and word2vec clarifications

I have documents, and I calculated the word vectors using word2vec for all the terms in my corpus. Now I want to compute similarity between documents using the bag-of-words model. After creating ...
3
votes
1answer
3k views

Word2Vec embeddings with TF-IDF

When you train the word2vec model (using for instance, gensim) you supply a list of words/sentences. But there does not seem to be a way to specify weights for the words calculated for instance using ...
1
vote
1answer
160 views

Word2Vec, softmax function

I was going term by term through the softmax function for the word2vec (SKIP-GRAM) model. I found most definition of these functions to be not 'clear' so I modified the notation to make sure I ...
1
vote
0answers
189 views

Use pretrained word vectors over custom trained word2vecs

Currently i'm working on a sentiment analysis research project using LSTM networks. As the input I convert sentences into set of vectors using word2vec. And there are some well pretrained word ...
1
vote
1answer
43 views

Help me understand how word-as-vector representations are constructed

Let's suppose I have a big list of words. I want to turn this list into a vector space of dimension $N$ such that each word is a vector in this vector space. But I have no idea how to go about with ...
1
vote
0answers
116 views

How much text is enough to train a good embedding model?

I need to train a word2vec embedding model on Wikipedia articles using gensim. Eventually, I will use the entire Wikipedia for that but for the moment, I'm doing some experimentation/optimization to ...
3
votes
1answer
497 views

Training Doc2Vec and Word2Vec at the same time

As far as I can tell the typical Doc2Vec implementation (e.g. Gensim) first trains the word vectors and afterwards the document vectors were the word vectors are fixed. If my goal is that ...
2
votes
1answer
149 views

Does it make sense that datetime encodes one-hot-vector like one-hot-encoding or something else like

I'm new to machine learning and deep learning. I've wanted to solve time series problem, which has data every single second. Plus, I've been doing research on word2vector and time series data lately. ...