10 votes

NN embedding layer

At theoretical level, the embedding layer is a linear layer, there is not any difference at all. However, in practice, if you are building a deep learning software, you have to make a difference ...
David Masip's user avatar
  • 6,031
4 votes

NN embedding layer

The embedding layer maps your vocabulary index input to a dense vector, so it acts as lookup layer and (if set to trainable) will be influenced on some weights only, by the words occurring in a batch ...
Elliot's user avatar
  • 1,081
3 votes

ways to represent document by its keyword vectors

If you have the vectors for your keywords, you can aggregate those to get the document vectors. The simplest(and probably one of the most effective) way is to average out your keyword vectors to form ...
Gyan Ranjan's user avatar
2 votes

Can 2 different OOV words get the same vector in FastText?

TL;DR Theoretically it is possible, but it is unlikely. 1) Uncommon subwords ...
Bruno Lubascher's user avatar
2 votes

ways to represent document by its keyword vectors

It sounds like you're trying to do some sort of topic modeling. I might recommend something like Latent Semantic Analysis (LSA), Latent Dirichlet allocation (LDA), or Latent Semantic Indexing (LSI). ...
Alex L's user avatar
  • 371
2 votes

How to vectorize newline \n in tensorflow textVectorization layer?

Did you look at the tokens that are used during the vectorization? Maybe the '\n' character is not considered as a token. Thus it is not vectorized. That is why while trying to get back to the initial ...
1001pepi's user avatar
2 votes

Machine learning - Algorithm suggestion for my problem using NLP

This looks like a proper task for Doc2Vec, which is an algorithm to build paragraph embeddings. For a nice implementation with usage examples you can try out gensim. Other options can be using ...
TitoOrt's user avatar
  • 1,852
2 votes

Machine learning - Algorithm suggestion for my problem using NLP

There is a really good video about this topic from a PyCon in 2016. There is a pretty in-depth description on how to vectorize your sentences as well as make predictions based on those vectors. I ...
driftwood14's user avatar
1 vote

Scoring vectors by "distinctness"

Multiple methods come to mind. Here are a few ideas. The first one is very easy to implement and the other ones are clustering methods, for which many implementations exist already: Use cosine ...
qmeeus's user avatar
  • 1,259
1 vote

Similarity with respect to a specific concept in text embeddings

You can generate custom embeddings for your corpus/dataset and then calculate the cosine similarity. When you generate your own word embeddings for your dataset, words with similar meaning will be ...
YadneshD's user avatar
  • 111
1 vote

Dimensionality reduction of vectors with null values

You have several options: Drop rows that have null values. Impute the null values. Pick a dimensionality reduction algorithm that can handle null values. One example is NIPALS (Nonlinear Iterative ...
Brian Spiering's user avatar
1 vote

Word2Vec: Identifying many-to-one relationships between words

Those vector relations are not exact. Rest assured that king - queen ≠ man - woman. What we do is finding the closest vectors to the result of ...
noe's user avatar
  • 23.8k
1 vote

Non-commutative distance formula

Take for example the Euclidean distance L2, defined by: $$L_2(x,y) = \left(\sum_{i=1}^{d} (x_i-y_i)^2\right)^{1/2}$$ where $d$ is the vector dimension. You can easily add a term $\alpha \in (0,1)$ and ...
rapaio's user avatar
  • 4,643
1 vote

How come same cluster category be separated?

This KMeans Clustering is a representative of points in the space based on 200 Features and their length. I think there is some gap in your perspective and the actual clustering. Other than "...
10xAI's user avatar
  • 5,534
1 vote

Is it accurate to say that "K-means clustering the vectors based on keywords weight similarity"?

K-means clusters points (in your case represented as 745 dimensional vectors) by their similarity, that is some distance measure between points (usually the Euclidean distance). TF-IDF produces a ...
Tinu's user avatar
  • 508
1 vote

what is the difference between positional vector and attention vector used in transformer model?

So the question asks between the difference between an attention vector and a positional vector. To answer this question, will give some context into how the transformer differs from a sequential ...
shepan6's user avatar
  • 1,408
1 vote

How to dual encode two sentences to show similarity score

How can I create a dual encoder though? Do I use two different neural networks? Or does output just contain one neuron that outputs similarity? You don't need to create two different neural ...
Caxton's user avatar
  • 161
1 vote

Can I sum up feature vectors of a user‘s collection?

You can use total sum of boolean values. That will be fast and give a general notion of similarity. A more useful metric might be Hamming distance, the sum of matching booleans between two vectors.
Brian Spiering's user avatar
1 vote

What are the main distribution semantics based algorithms?

The main idea here is: "birds of a feather flock together" That is, words that appear near each other inform the "function" of a word. More importantly, I think of the techniques you mentioned as "...
ngopal's user avatar
  • 81
1 vote

Machine learning - Algorithm suggestion for my problem using NLP

You need to do fit_transform first then transform, Here sample example ...
Rakesh Chaudhari's user avatar
1 vote

Stacking/Concatenating/Combining two vector space models

It is perfectly valid to concatenate the vectors from two different models. It will be necessary however to experiment with the two approaches, i.e. either using individual vectors from one of the ...
Sam  - Founder of AceAINow.com's user avatar
1 vote

Collection Of Variable Length Sequences and Descriptions: A Search Problem

While an RNN using one-hot encoded moves is possible, I would suggest that your model needs to understand chess (or similar complex games) at a deeper level to be able to associate comments to ...
Pavel Savine's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible