10
votes
NN embedding layer
At theoretical level, the embedding layer is a linear layer, there is not any difference at all. However, in practice, if you are building a deep learning software, you have to make a difference ...
- 6,031
4
votes
NN embedding layer
The embedding layer maps your vocabulary index input to a dense vector, so it acts as lookup layer and (if set to trainable) will be influenced on some weights only, by the words occurring in a batch ...
- 1,081
3
votes
ways to represent document by its keyword vectors
If you have the vectors for your keywords, you can aggregate those to get the document vectors. The simplest(and probably one of the most effective) way is to average out your keyword vectors to form ...
- 801
2
votes
Accepted
Can 2 different OOV words get the same vector in FastText?
TL;DR
Theoretically it is possible, but it is unlikely.
1) Uncommon subwords
...
- 3,518
2
votes
ways to represent document by its keyword vectors
It sounds like you're trying to do some sort of topic modeling. I might recommend something like Latent Semantic Analysis (LSA), Latent Dirichlet allocation (LDA), or Latent Semantic Indexing (LSI). ...
- 371
2
votes
How to vectorize newline \n in tensorflow textVectorization layer?
Did you look at the tokens that are used during the vectorization? Maybe the '\n' character is not considered as a token. Thus it is not vectorized. That is why while trying to get back to the initial ...
- 21
2
votes
Machine learning - Algorithm suggestion for my problem using NLP
This looks like a proper task for Doc2Vec, which is an algorithm to build paragraph embeddings. For a nice implementation with usage examples you can try out gensim.
Other options can be using ...
- 1,852
2
votes
Machine learning - Algorithm suggestion for my problem using NLP
There is a really good video about this topic from a PyCon in 2016. There is a pretty in-depth description on how to vectorize your sentences as well as make predictions based on those vectors.
I ...
- 23
1
vote
Scoring vectors by "distinctness"
Multiple methods come to mind. Here are a few ideas. The first one is very easy to implement and the other ones are clustering methods, for which many implementations exist already:
Use cosine ...
- 1,259
1
vote
Similarity with respect to a specific concept in text embeddings
You can generate custom embeddings for your corpus/dataset and then calculate the cosine similarity. When you generate your own word embeddings for your dataset, words with similar meaning will be ...
- 111
1
vote
Dimensionality reduction of vectors with null values
You have several options:
Drop rows that have null values.
Impute the null values.
Pick a dimensionality reduction algorithm that can handle null values. One example is NIPALS (Nonlinear Iterative ...
- 20.5k
1
vote
Word2Vec: Identifying many-to-one relationships between words
Those vector relations are not exact. Rest assured that king - queen ≠ man - woman. What we do is finding the closest vectors to the result of ...
- 23.8k
1
vote
Non-commutative distance formula
Take for example the Euclidean distance L2, defined by:
$$L_2(x,y) = \left(\sum_{i=1}^{d} (x_i-y_i)^2\right)^{1/2}$$
where $d$ is the vector dimension. You can easily add a term $\alpha \in (0,1)$ and ...
- 4,643
1
vote
How come same cluster category be separated?
This KMeans Clustering is a representative of points in the space based on 200 Features and their length. I think there is some gap in your perspective and the actual clustering.
Other than "...
- 5,534
1
vote
Is it accurate to say that "K-means clustering the vectors based on keywords weight similarity"?
K-means clusters points (in your case represented as 745 dimensional vectors) by their similarity, that is some distance measure between points (usually the Euclidean distance).
TF-IDF produces a ...
- 508
1
vote
Accepted
what is the difference between positional vector and attention vector used in transformer model?
So the question asks between the difference between an attention vector and a positional vector.
To answer this question, will give some context into how the transformer differs from a sequential ...
- 1,408
1
vote
How to dual encode two sentences to show similarity score
How can I create a dual encoder though? Do I use two different neural
networks? Or does output just contain one neuron that outputs
similarity?
You don't need to create two different neural ...
- 161
1
vote
Accepted
Can I sum up feature vectors of a user‘s collection?
You can use total sum of boolean values. That will be fast and give a general notion of similarity.
A more useful metric might be Hamming distance, the sum of matching booleans between two vectors.
- 20.5k
1
vote
What are the main distribution semantics based algorithms?
The main idea here is: "birds of a feather flock together" That is, words that appear near each other inform the "function" of a word.
More importantly, I think of the techniques you mentioned as "...
- 81
1
vote
Machine learning - Algorithm suggestion for my problem using NLP
You need to do fit_transform first then transform, Here sample example
...
- 151
1
vote
Stacking/Concatenating/Combining two vector space models
It is perfectly valid to concatenate the vectors from two different models. It will be necessary however to experiment with the two approaches, i.e. either using individual vectors from one of the ...
1
vote
Collection Of Variable Length Sequences and Descriptions: A Search Problem
While an RNN using one-hot encoded moves is possible, I would suggest that your model needs to understand chess (or similar complex games) at a deeper level to be able to associate comments to ...
- 571
Only top scored, non community-wiki answers of a minimum length are eligible