9 votes

NN embedding layer

At theoretical level, the embedding layer is a linear layer, there is not any difference at all. However, in practice, if you are building a deep learning software, you have to make a difference ...
  • 5,714
4 votes

NN embedding layer

The embedding layer maps your vocabulary index input to a dense vector, so it acts as lookup layer and (if set to trainable) will be influenced on some weights only, by the words occurring in a batch ...
  • 1,071
3 votes

ways to represent document by its keyword vectors

If you have the vectors for your keywords, you can aggregate those to get the document vectors. The simplest(and probably one of the most effective) way is to average out your keyword vectors to form ...
2 votes

ways to represent document by its keyword vectors

It sounds like you're trying to do some sort of topic modeling. I might recommend something like Latent Semantic Analysis (LSA), Latent Dirichlet allocation (LDA), or Latent Semantic Indexing (LSI). ...
  • 361
2 votes
Accepted

Can 2 different OOV words get the same vector in FastText?

TL;DR Theoretically it is possible, but it is unlikely. 1) Uncommon subwords ...
2 votes

Machine learning - Algorithm suggestion for my problem using NLP

This looks like a proper task for Doc2Vec, which is an algorithm to build paragraph embeddings. For a nice implementation with usage examples you can try out gensim. Other options can be using ...
  • 1,772
2 votes

Machine learning - Algorithm suggestion for my problem using NLP

There is a really good video about this topic from a PyCon in 2016. There is a pretty in-depth description on how to vectorize your sentences as well as make predictions based on those vectors. I ...
1 vote

Dimensionality reduction of vectors with null values

You have several options: Drop rows that have null values. Impute the null values. Pick a dimensionality reduction algorithm that can handle null values. One example is NIPALS (Nonlinear Iterative ...
1 vote

Word2Vec: Identifying many-to-one relationships between words

Those vector relations are not exact. Rest assured that king - queen ≠ man - woman. What we do is finding the closest vectors to the result of ...
  • 16.3k
1 vote

Non-commutative distance formula

Take for example the Euclidean distance L2, defined by: $$L_2(x,y) = \left(\sum_{i=1}^{d} (x_i-y_i)^2\right)^{1/2}$$ where $d$ is the vector dimension. You can easily add a term $\alpha \in (0,1)$ and ...
  • 4,448
1 vote

How come same cluster category be separated?

This KMeans Clustering is a representative of points in the space based on 200 Features and their length. I think there is some gap in your perspective and the actual clustering. Other than "...
  • 5,224
1 vote

Is it accurate to say that "K-means clustering the vectors based on keywords weight similarity"?

K-means clusters points (in your case represented as 745 dimensional vectors) by their similarity, that is some distance measure between points (usually the Euclidean distance). TF-IDF produces a ...
  • 478
1 vote
Accepted

what is the difference between positional vector and attention vector used in transformer model?

So the question asks between the difference between an attention vector and a positional vector. To answer this question, will give some context into how the transformer differs from a sequential ...
  • 1,321
1 vote

How to dual encode two sentences to show similarity score

How can I create a dual encoder though? Do I use two different neural networks? Or does output just contain one neuron that outputs similarity? You don't need to create two different neural ...
  • 161
1 vote

Can I sum up feature vectors of a user‘s collection?

You can use total sum of boolean values. That will be fast and give a general notion of similarity. A more useful metric might be Hamming distance, the sum of matching booleans between two vectors.
1 vote

What are the main distribution semantics based algorithms?

The main idea here is: "birds of a feather flock together" That is, words that appear near each other inform the "function" of a word. More importantly, I think of the techniques you mentioned as "...
  • 81
1 vote

Machine learning - Algorithm suggestion for my problem using NLP

You need to do fit_transform first then transform, Here sample example ...
1 vote

Stacking/Concatenating/Combining two vector space models

It is perfectly valid to concatenate the vectors from two different models. It will be necessary however to experiment with the two approaches, i.e. either using individual vectors from one of the ...
1 vote

Collection Of Variable Length Sequences and Descriptions: A Search Problem

While an RNN using one-hot encoded moves is possible, I would suggest that your model needs to understand chess (or similar complex games) at a deeper level to be able to associate comments to ...

Only top scored, non community-wiki answers of a minimum length are eligible