New answers tagged word-embeddings
0
I recommend this paper. The authors treat the size of embeddings as a hyperparameter and provide a detailed study on it. They show that this dimensionality should depend on the corpus.
1
You cannot run the universal sentence encoder in reverse. There is no practical way to take an arbitrary embedding vector and get a sentence.
My suggestion would instead be to find the sentence in your data with the embedding closest to your center. Euclidean distance works well, specially if you used K-means or another euclidean method to create your ...
1
Welcome to the community,
I do not know about other libraries, but gensim has a very good API to create word2vec models. In order to preprocess data, you have to decide first what things you are gonna keep in your vocab and whatnot. for ex:- Punctuations, numbers, alphanumeric words(ex - 42nd) etc.
In my knowledge, the most generic preprocessing pipeline ...
1
It depends upon where do you want to submit your results that you claim, and what is the submission criteria.
First, it is unclear if "lower error in regression" is training or validation/test error. I am sure that letting Embedding layer to be trainable will adapt it to fit the training set better, but it might cause overfitting and cause higher error on ...
1
So in the end I dealt with this as a latent semantic analysis problem by concatenating the array of category descriptions into a single long string, then passing that through sklearn's TfIdfTransformer and TruncatedSVD. This worked fine, although the neural network I had already built outperformed any sklearn algorithm I tried.
3
Some points first:
BERT is a word embedding: BERT is both word and sentence embedding. It needs to be taken into account that BERT is taking the sequence of words in a sentence into account which gives you a richer embedding of words in a context but in classic embeddings (yes, after BERT we can call others "classic"!) you mostly deal with neighborhood i.e. ...
0
When you run BERT, you get one vector per input token + 1 special token called [CLS] + 1 special token called [SEP]. Maybe more precise than calling BERT embeddings as embeddings, would be calling them hidden states of BERT. The contextual information get into the embeddings via 12 layers of self-attentive neural network.
However, the tokenization is tricky ...
4
BERT generates contextualized word embeddings, which means that BERTprovides the most accurate embeddings when a word is in a sentence(context).
For each of the words within the sentence, BERT will generate a vector of numbers. In your case, you will have a good representation of the word "bank".
So if you have a sentence for all the other words that you ...
0
It depends on what information you want to capture :
If you want to capture the passing of time, encoding your date as days since a reference date might be a good idea.
If you want to encode the cyclicality of time (months in a year), you can encode your month variables on a circle.
2
It depends…
The general rule of thumb is that there should be at least 40 occurrences of an item to train an embedding model to find a robust representation. If most follower IDs repeat then an embedding model can learn which ones co-occur. If follower IDs are sparse then hashing (which randomly assigns numbers) is a better choice.
Which method is better ...
Top 50 recent answers are included
