# Questions tagged [cosine-distance]

A measure of the angular distance between two vectors. Usually defined as 1-(cosine similarity).

69 questions
Filter by
Sorted by
Tagged with
8 views

### Inner working behind combining two distance function as one function for similarity measure

I am comparing two images and for this I am testing various similarity function. For my case, Euclidean works much better than cosine(20% difference). However, I tried to combine two distance function ...
18 views

### String Matching

I have to work on 2 datasets where I have to find out the duplicates in addresses present in both the dataset. I am a bit confused that which one of the Levenshtein distance or cosine similarity, I ...
8 views

### Results interpretation of AgglomerativeClustering labelling

First of all I would like to say that I'm quite new to python and even more new to scikit, and I'm also a self learner, so please forgive my banal question, but it doesn't look banal to me. So, I have ...
41 views

### Cosine vs Manhattan for Text Similarity [closed]

I'm storing sentences in Elasticsearch as dense_vector field and used BERT for the embedding so each vector is 768 dim. ...
19 views

### Matching documents from different sets with tfidf and cosine distance

I have two different set of documents S1, S2, with 30 text documents each. Using some text representation method, such as tfidf ...
34 views

### how to calculate the cosine similarity between two files?

I am using spark and scala to implement an issue. files contain phrases or sentences. I want to use domain based method to calculate the cosine similarity between tags.I convert two files into a ...
69 views

### pairwise_distances with Cosine and weighting

Is there a way to get a weight into the pairwise_distances(X, metric='cosine') Potentially using **kwrds? ...
20 views

### Is there a way to calculate the cosine distance between 2 time series?

Let's say I have time series data of City A, City B, City C & City D that looks like this: ...
174 views

### Calculate the similarity between pairs of time series data

I have 5 pieces of time series data. It is the weekly sales of 5 different stores (A,B,C,D,E). There are no missing values. A quick visual inspection shows that these 5 pieces of time series data have ...
54 views

### Is normalizing term weight necessary when cosine similarity is used in retrieval?

When using cosine similarity in information retrieval, document vector length and query vector length are used for normalization. So if TF-IDF is used as a weighting function, then using raw frequency ...
79 views

### Cosine Similarity but with weighting for vector indexes

I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online. This is my problem: I have extracted and ranked keywords from 2 text sources: A ...
77 views

### Evaluate document similarity / content-based recommender system

I'm planning on building a basic content-based recommender system with word2vec and cosine similarity. The data consists of 300k documents in varying length. How do I evaluate my model if I have no ...
30 views

### finding similarity of a new datapoint

I have built a recommendation engine using cosine similarity. When I want to find all the records similar to a given record that is already present in the dataset it works. Consider a case, a user ...
36 views

### Item-to-Item recommendation using DNN

I am new to DNN still learning, have a need to build item-to-item content based recommendation using DNN. For example, say I have a column of strings where each row represents a document I need to ...
179 views

### If i use use BERT embeddings for if cosine(sent1,sent2) > 0.9, then is it fair to assume s1 and s2 are similar

According to BERT author Jacob Devlin: I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens ...
42 views

### Is the magnitude of a word vector correlated with the frequency of the word in a text?

In order to find the similarity between words, cosine similarity seems to be the most common measure to use. In a conversation that I had about this topic, someone mentioned that words that mean more ...
181 views

### Question about BERT embeddings with high cosine similarity

Under what circumstances would BERT assign two occurrences of the same word similar embeddings? If those occurrences are contained within similar syntactic relations with their co-occurrents?
607 views

### Semantic similarity between two or more sentences

I need to determine how similar sentences (in meaning) are to one another. In order to do it, I have been considering an algorithm (cosine similarity) to determine the similarity between sentences. I ...
1k views

### Why is the cosine distance used to measure the similatiry between word embeddings?

While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean ...
42 views

### Is summing a cosine similarity matrix a good way to determine overall similarity?

I'm trying to similar research abstracts, so I'm using word embeddings to convert words into 1x768 vectors, so overall turning abstracts into embeddings with shape (#ofwords, 768). Cosine similarity ...
373 views

19 views

### TensorFlow: CosineDifference ObjFunc Constant throughout training

The following example is a simplified version of what I'm working on. I'm trying to find a neural network which minimises the cosine distance. The reason I have implemented my own cosine difference ...
3k views

### Cosine similarity vs The Levenshtein distance

Cosine similarity vs The Levenshtein distance I wanted to know what is the difference between them and in what situations they work best? As per my understanding: Cosine similarity is a measure of ...
78 views

### Siamese networks vs Semantic similarity (may be gensim)

I am trying to understand the Siamese networks . In this vector is calculated for an object (say an image) and a distance metric is applied (say manhatten) on two vectors produced by the neural ...
339 views

### counter vector fit transform cosine similarity memory error

count_matrix = count.fit_transform(off_data3['bag_of_words']) I have count_matrix shape with count_matrix.shape (476147, 482824) ...
4k views

### memory error in matrix cosine_similarity

I have (20905040, 7) of a dataset to recommend 10 different product to the user it could be larger than that but anyway I got memory error when processing the ...
469 views

### Elbow method for cosine distance

I have clustered vectors by cosine distance using nltk clusterer. If I understand correctly, Y axis for elbow method in euclidian distance would be the sum of every distance (squared) between centroid ...
41 views

### Document matching with more priority to certain features than others

I am working on recommendation systems wherein I need to match the similarity of 2 users. Now, I know that I can use Tfidf vectorizer to calculate the the cosine similarity score between them. But, ...
4k views

### Calculating cosine similarity between 3D arrays using Python

I have two matrices with multiple columns and three rows each. I calculated the cosine similarity (sklearn) but it gives the result as a matrix. How can I obtain one single value? The matrices are the ...
59 views

### Checking TF-IDF Results

I am working with TF-IDF and cosine similarity to do document comparisons and given a document, which document in the data is the most similar. However, sometimes it returns a high similarity between ...
910 views

### word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...
6k views

### Hierarchical clustering with precomputed cosine similarity matrix using scikit learn produces error

We want to use cosine similarity with hierarchical clustering and we have cosine similarities already calculated. In the sklearn.cluster.AgglomerativeClustering documentation it says: A distance ...
2k views

### Cosine similarity with arrays contaning NaN

I am trying to calculate a cosine similarity using Python in order to find similar users basing on ratings they have given to movies. As it can be expected there are a lot of NaN values. I am using ...
4k views

### how to create word2vec for phrases and then calculate cosine similarity

I have just started using word2vec and I have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate cosine similarity between ...
109 views

### Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
40 views

### When I would use a specific similarity coefficient over another?

Like using Jaccards over Dice. I want real examples, of when I would prefer to use Jaccards, Dice, Cosine or any other similarity coefficient.
2k views

### cosine similarity between items (purchase data) and normalisation

I'm using IndexedRowMatrix which represents the products's user purchase behaviours and in order to build product recommendations, I use cosine similarity to calculate similarities between products. ...
34 views

### Best way to find dissimilarity in a 6x2 DataFrame?

I'm new to data science and am currently learning different techniques that I can do with Python. Currently, I'm trying it out with Spotify's API for my own playlists. The goal is to find the most ...
33 views

### Can I sum up feature vectors of a user‘s collection?

I want to find items that are similar to items users already have in their collection. Every item has attributes, so I created feature vectors where every element of the vector represents an attribute ...
802 views

### clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...