Skip to main content
Share Your Experience: Take the 2024 Developer Survey
3 votes

How do we evaluate the outputs of text generation models?

Well, you missed the old good human evaluation, which is the only actual measure that can actually be trusted in terms of semantic evaluation. Also, in the reference n-gram matching area you missed ...
noe's user avatar
  • 26.9k
3 votes
Accepted

Threshold determination / prediction for cosine similarity scores

As far as I know there is no satisfactory answer: One uses a threshold in order to avoid having to choose a specific K in a top K approach. The threshold is often selected manually to eliminate the ...
Erwan's user avatar
  • 25.5k
3 votes
Accepted

How to build recommendation model based on resume and job description?

If you want a DL approach, I recommend substituting the tf-idf by some kind of word embeddings. For instance, you can take a pre-trained word embedding model, like glove, and average its outputs both ...
David Masip's user avatar
  • 6,081
2 votes
Accepted

What's the best way to generate similar words?

But the alternative is to navigate a treebank, through "type of" relationships… much, much faster and cheaper. WordNet provides exactly this: it is a lexical database in which words are ...
Erwan's user avatar
  • 25.5k
2 votes
Accepted

Semantic similarity between two or more sentences

Word2vec as the name suggests will create an embedding for each word in your sentence. In order to get a sentence level embedding you would need to average (or combine in some other way) the ...
Brandon Donehoo's user avatar
2 votes
Accepted

How to choose similarity measurement between sentences and paragraphs

Simphile NLP package creator here. Choosing the text similarity method that works best for your application can be difficult. Ideally you have a large sample set with many known positives (i.e. text ...
Brian Risk's user avatar
2 votes
Accepted

Cosine vs Manhattan for Text Similarity

Intuitively, if you normalized the vectors before using them, or if they all ended up having almost unit norm after training, then a small $l_1$ norm will imply that the angle between the vectors is ...
Miguel's user avatar
  • 355
2 votes

Why do semantically different words produce similar embeddings?

I explored this very problem in one of my medium posts.Why are Cosine similarities almost always positive. Quoting from that: In other words, the cosine similarity has a positive contribution if the ...
Vaibhav Garg's user avatar
2 votes

Is there a reference dataset for contextual similarity?

I think some datasets used for word sense disambiguation (WSD) would be an option. WSD is the task of classifying an ambiguous word into its correct meaning. For instance "apple" would have ...
Erwan's user avatar
  • 25.5k
2 votes

where to store embeddings for similarity search?

There's Milvus search engine that utilizes several prominent Approximate KNN libraries such as FAISS, ANNOY and HNSW. It also handles several bookkeeping, clustering, data integrity and other tasks ...
SimpleV's user avatar
  • 121
2 votes
Accepted

How to combine two vector embeddings into one?

How do you plan on using these embeddings? You can definitely use concatenated embeddings for similarity/retrieval, but only when comparing concat embeddings to other concat embeddings. Your point ...
Karl's user avatar
  • 756
1 vote
Accepted

How does RAG query affect the similarity search?

The query is entered in question form because it is easy. There are different variations of what you embed to compute the similarity search. Another variation is to let a LLM hallucinate a few answers ...
noe's user avatar
  • 26.9k
1 vote

Visualizing Author Topic Similarities: t-SNE and Cluster Labeling

My baseline for this task is as follows (to analyze the abstract): tokenize abstract, normalize tokens, vectorize tokens (FastText, BERT, ...), add vectors, cluster with DBSCAN, changing threshold ...
Gromov's user avatar
  • 126
1 vote

Semantic Search on numeric data

You could feed the LLM a description of the file format and then request it to generate a piece of code to extract the information you want, for instance, in Python. Then, you would run the generated ...
noe's user avatar
  • 26.9k
1 vote

What do averaged word vectors represent?

Many questions on a single post, let's go step by step: Assume you have taken the average of all vectors of words referring to animals (like family and species names, common terms like "cat"...
Memristor's user avatar
  • 256
1 vote

Deduplication using NLP

You could try the following: Similarity Metrics: There are several similarity metrics that can be used to compare the similarity between two texts. For example, the Jaccard similarity coefficient, ...
Vinita Silaparasetty's user avatar
1 vote

Semantic search - combine text and image embedding

Currently I am working in a marketplace too, and I am trying to combine text and embedding features. I am doing this by simply concatenation of them. It's important to normalize the text features and ...
Фёдор Курушин's user avatar
1 vote

Document Similarity with User Preference

These are two different things: document similarity is based only on the documents previous users choices can be used to train a recommender system, or simply applied in a rule-based fashion. The ...
Erwan's user avatar
  • 25.5k
1 vote

Cluster words into groups of similar meaning (synonyms)

It is not possible to find synonyms by starting with word embeddings. Word embeddings group words by co-occurrence. For example "and" and "but" will be near each other in embedding ...
Brian Spiering's user avatar
1 vote
Accepted

Document Content

If you are asking how to integrate this, I would leverage existing search technologies such as storing documents in mongo database or using ...
eliangius's user avatar
  • 351
1 vote

applicability of relative similarity computation

It's often useful to think of simple cases, e.g. even in a 2-D (planar) case, you can't determine z. Similarity between two vectors is identical to the angle (at the origin) between them, so: for a ...
Carl's user avatar
  • 396
1 vote
Accepted

BERT Optimization for Production

you can start by using torchscript, it may require changing ur whole code, and switching to transformers( by loading the backbone of the model and the last layers) so basically u get out from GIL ...
Simone's user avatar
  • 242
1 vote
Accepted

Semantic networks: word2vec?

There are a few models that are trained to analyse a sentence and classify each token (or recognise dependencies between words). Part of speech tagging (POS) models assign to each word its function (...
RonsenbergVI's user avatar
  • 1,024
1 vote

Modelling for similarity between two descriptions

So the questions asks for how to compute similarity between the organisation description and project titles. One initial thought would be to use a Doc2Vec model (concept, implementation), which will ...
shepan6's user avatar
  • 1,438
1 vote

NLP: Compare tags semantically with machine learning? (finding synonyms)

You can use word embeddings take a look at word2vec and glove. There are many good tutorials on word2vec, but very briefly word2vec represents a word as a vector, so that you can do a similarity ...
Akavall's user avatar
  • 924
1 vote
Accepted

How to determine whether a semantic concept is present in a string

This is essentially information retrieval: usually there is a collection of documents and the goal is to find the document which is the most similar to a given query (what you call the "semantic ...
Erwan's user avatar
  • 25.5k
1 vote
Accepted

Similarity Threshold Standards

I don't think there's any standard, but there might be some exceptions in very specific cases where the distribution of the scores is known precisely. There's no standard because in general the ...
Erwan's user avatar
  • 25.5k
1 vote

Sentence to word similarity

If you have a large corpus of text mapped with their respective topics, you can train a Siamese neural network where you have two inputs (one sentence and one topic) and output a similarity score ...
AlexPnt's user avatar
  • 179
1 vote

Sentence to word similarity

If you have a list of words for every topic you can indeed try to directly measure the similarity of this list against a sentence, but it's likely that a sentence doesn't always contain one of the ...
Erwan's user avatar
  • 25.5k
1 vote

Evaluation metric for Information retrieval system

I want to know how to come up with ground truth(relevancy label) if it's not available? There's simply no way to properly evaluate a system if nobody knows what the output is supposed to be. However ...
Erwan's user avatar
  • 25.5k

Only top scored, non community-wiki answers of a minimum length are eligible