Newest 'cosine-distance' Questions

0

votes

0answers

8 views

Inner working behind combining two distance function as one function for similarity measure

I am comparing two images and for this I am testing various similarity function. For my case, Euclidean works much better than cosine(20% difference). However, I tried to combine two distance function ...

asked yesterday

Rambo_john

134 bronze badges

1

vote

1answer

18 views

String Matching

I have to work on 2 datasets where I have to find out the duplicates in addresses present in both the dataset. I am a bit confused that which one of the Levenshtein distance or cosine similarity, I ...

cosine-distance

asked 2 days ago

Debi Prasad Sen

111 bronze badge

0

votes

0answers

8 views

Results interpretation of AgglomerativeClustering labelling

First of all I would like to say that I'm quite new to python and even more new to scikit, and I'm also a self learner, so please forgive my banal question, but it doesn't look banal to me. So, I have ...

python scikit-learn clustering cosine-distance agglomerative

asked Jul 1 at 20:32

dao.foa

11 bronze badge

0

votes

1answer

41 views

Cosine vs Manhattan for Text Similarity [closed]

I'm storing sentences in Elasticsearch as dense_vector field and used BERT for the embedding so each vector is 768 dim. ...

cosine-distance similar-documents semantic-similarity manhattan

asked Jun 26 at 16:40

Mohy Mohamed

553 bronze badges

1

vote

0answers

19 views

Matching documents from different sets with tfidf and cosine distance

I have two different set of documents S1, S2, with 30 text documents each. Using some text representation method, such as tfidf ...

dataset tfidf cosine-distance similar-documents document-term-matrix

asked Apr 16 at 10:14

forgotten_novel_char

1112 bronze badges

1

vote

0answers

34 views

how to calculate the cosine similarity between two files?

I am using spark and scala to implement an issue. files contain phrases or sentences. I want to use domain based method to calculate the cosine similarity between tags.I convert two files into a ...

recommender-system apache-spark implementation cosine-distance scala

asked Feb 24 at 18:29

salm

112 bronze badges

1

vote

1answer

69 views

pairwise_distances with Cosine and weighting

Is there a way to get a weight into the pairwise_distances(X, metric='cosine') Potentially using **kwrds? ...

python scikit-learn cosine-distance

asked Feb 1 at 22:37

Rolfey

111 bronze badge

0

votes

0answers

20 views

Is there a way to calculate the cosine distance between 2 time series?

Let's say I have time series data of City A, City B, City C & City D that looks like this: ...

time-series similarity cosine-distance

asked Jan 21 at 14:22

DPatrick

1133 bronze badges

1

vote

1answer

174 views

Calculate the similarity between pairs of time series data

I have 5 pieces of time series data. It is the weekly sales of 5 different stores (A,B,C,D,E). There are no missing values. A quick visual inspection shows that these 5 pieces of time series data have ...

asked Jan 7 at 1:47

DPatrick

1133 bronze badges

0

votes

0answers

54 views

Is normalizing term weight necessary when cosine similarity is used in retrieval?

When using cosine similarity in information retrieval, document vector length and query vector length are used for normalization. So if TF-IDF is used as a weighting function, then using raw frequency ...

normalization tfidf cosine-distance vector-space-models

asked Dec 30 '20 at 17:44

Nina

1012 bronze badges

2

votes

1answer

79 views

Cosine Similarity but with weighting for vector indexes

I am very new to NLP and although this seems like a basic question I don't know how to search for an answer online. This is my problem: I have extracted and ranked keywords from 2 text sources: A ...

machine-learning nlp ranking cosine-distance

asked Dec 12 '20 at 14:24

K Kreid

211 bronze badge

1

vote

2answers

77 views

Evaluate document similarity / content-based recommender system

I'm planning on building a basic content-based recommender system with word2vec and cosine similarity. The data consists of 300k documents in varying length. How do I evaluate my model if I have no ...

recommender-system word2vec cosine-distance similar-documents

asked Nov 23 '20 at 15:15

jonas

434 bronze badges

1

vote

1answer

30 views

finding similarity of a new datapoint

I have built a recommendation engine using cosine similarity. When I want to find all the records similar to a given record that is already present in the dataset it works. Consider a case, a user ...

data-science-model recommender-system cosine-distance

asked Nov 2 '20 at 19:35

Raj

1494 bronze badges

1

vote

1answer

36 views

Item-to-Item recommendation using DNN

I am new to DNN still learning, have a need to build item-to-item content based recommendation using DNN. For example, say I have a column of strings where each row represents a document I need to ...

neural-network recommender-system cosine-distance

asked Oct 20 '20 at 16:04

Raj

1494 bronze badges

1

vote

1answer

179 views

If i use use BERT embeddings for if cosine(sent1,sent2) > 0.9, then is it fair to assume s1 and s2 are similar

According to BERT author Jacob Devlin: I'm not sure what these vectors are, since BERT does not generate meaningful sentence vectors. It seems that this is doing average pooling over the word tokens ...

nlp bert cosine-distance semantic-similarity

asked Oct 12 '20 at 13:16

user2478236

1133 bronze badges

1

vote

0answers

42 views

Is the magnitude of a word vector correlated with the frequency of the word in a text?

In order to find the similarity between words, cosine similarity seems to be the most common measure to use. In a conversation that I had about this topic, someone mentioned that words that mean more ...

nlp word-embeddings word2vec similarity cosine-distance

asked Sep 30 '20 at 9:48

Emil

1897 bronze badges

1

vote

1answer

181 views

Question about BERT embeddings with high cosine similarity

Under what circumstances would BERT assign two occurrences of the same word similar embeddings? If those occurrences are contained within similar syntactic relations with their co-occurrents?

nlp bert transformer cosine-distance

asked Sep 10 '20 at 15:13

Gog

132 bronze badges

0

votes

1answer

607 views

Semantic similarity between two or more sentences

I need to determine how similar sentences (in meaning) are to one another. In order to do it, I have been considering an algorithm (cosine similarity) to determine the similarity between sentences. I ...

python nlp word2vec cosine-distance semantic-similarity

asked Sep 9 '20 at 0:16

Math

1439 bronze badges

5

votes

1answer

1k views

Why is the cosine distance used to measure the similatiry between word embeddings?

While computing the similarity between the words, cosine similarity or distance is computed on word vectors. Why aren't other distance metrics such as Euclidean ...

word-embeddings distance cosine-distance

asked Sep 3 '20 at 12:45

Ashwin Geet D'Sa

6991 gold badge4 silver badges16 bronze badges

1

vote

0answers

42 views

Is summing a cosine similarity matrix a good way to determine overall similarity?

I'm trying to similar research abstracts, so I'm using word embeddings to convert words into 1x768 vectors, so overall turning abstracts into embeddings with shape (#ofwords, 768). Cosine similarity ...

nlp cosine-distance

asked Aug 26 '20 at 23:06

Brian Guan

111 bronze badge

-1

votes

1answer

373 views

Matrix of pairwise cosine similarities from matrix of vectors [closed]

I have a matrix of 200d vectors stored as follows: $ X = \begin{pmatrix} \text{id}_1 & 0.5 & -2 & \dots & 10 \\ \text{id}_2 & -4 & 6 & \dots & -0.3 ...

pandas data numpy cosine-distance

asked Jul 24 '20 at 12:40

user11128

99

2

votes

1answer

239 views

How to get the probability/closeness of a sample belonging to a specific cluster?

I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect I got a set of ...

classification clustering unsupervised-learning k-means cosine-distance

asked Jun 13 '20 at 19:51

Jaskaran Singh Puri

1433 bronze badges

1

vote

0answers

21 views

Should I create a tfidf on a subset of a data set or use the whole corpus?

My goal in this project is to see if businesses on a list are currently customers within my organization. One piece of this involves producing a similarity score using cosine similarity on the names ...

python scikit-learn tfidf cosine-distance

asked Apr 13 '20 at 15:33

Ian

1112 bronze badges

1

vote

3answers

785 views

Fastest way for 1 vs all lookup on embeddings

I have a dataset with about 1 000 000 texts where I have computed their sentence embeddings with a language model and stored them in a numpy array. I wish to compare a new unseen text to all the 1 ...

machine-learning bert embeddings cosine-distance similar-documents

asked Mar 15 '20 at 15:29

Isbister

1536 bronze badges

0

votes

1answer

224 views

How fit_transform, transform and TfidfVectorizer works

I'm a machine learning beginner and I tried to use the cosine similarity on fuzzy matching purpose. In the following example I want to compare 'data_dirty' with 'data_clean' : When I have to ...

machine-learning python scikit-learn cosine-distance fuzzy-logic

asked Mar 11 '20 at 23:50

nananinanana

1

vote

0answers

22 views

NearestNeighbors testing

I have used nbrs = NearestNeighbors(metric= 'cosine', algorithm='brute').fit(items_features) distances, indices = nbrs.kneighbors(item_features) to find some ...

classification data bigdata cosine-distance

asked Feb 27 '20 at 16:19

AFB

411 silver badge4 bronze badges

1

vote

1answer

84 views

Understanding cosine distance with word vectors

I'm a new DL4J user, and I'm running all the works of Shakespeare through a Word2Vec neural net. I've got a pretty basic question about how to understand the results so far. In the below example, ...

nlp cosine-distance

asked Jan 19 '20 at 19:12

DPCII

354 bronze badges

2

votes

1answer

94 views

Approach to semantic similarity between documents

I was wondering what approach people would take, or point me in the right direction on this challenge I have set myself. I am pretty new at this, I have covered some area but want to expand my ...

nlp similarity cosine-distance doc2vec

asked Jan 8 '20 at 13:39

user5067291

1212 bronze badges

5

votes

1answer

86 views

Evaluating the performance of a machine learned recommendation system

I have a set of resumes $R=\{{r_1,...,r_n\}}$, which I've transformed to a vector space using TF-IDF. Each resume has a label, which is the name of their current employer. Each of these labels comes ...

machine-learning recommender-system evaluation information-retrieval cosine-distance

asked Dec 6 '19 at 22:27

Data

4273 silver badges11 bronze badges

0

votes

1answer

19 views

TensorFlow: CosineDifference ObjFunc Constant throughout training

The following example is a simplified version of what I'm working on. I'm trying to find a neural network which minimises the cosine distance. The reason I have implemented my own cosine difference ...

deep-learning tensorflow cosine-distance

asked Nov 24 '19 at 12:09

Iain MacCormick

335 bronze badges

4

votes

3answers

3k views

Cosine similarity vs The Levenshtein distance

Cosine similarity vs The Levenshtein distance I wanted to know what is the difference between them and in what situations they work best? As per my understanding: Cosine similarity is a measure of ...

similarity metric cosine-distance

asked Nov 18 '19 at 8:52

Pluviophile

1,5873 gold badges15 silver badges38 bronze badges

0

votes

1answer

78 views

Siamese networks vs Semantic similarity (may be gensim)

I am trying to understand the Siamese networks . In this vector is calculated for an object (say an image) and a distance metric is applied (say manhatten) on two vectors produced by the neural ...

cnn gensim cosine-distance siamese-networks semantic-similarity

asked Oct 23 '19 at 4:57

Sandeep Bhutani

7243 silver badges17 bronze badges

0

votes

1answer

339 views

counter vector fit transform cosine similarity memory error

count_matrix = count.fit_transform(off_data3['bag_of_words']) I have count_matrix shape with count_matrix.shape (476147, 482824) ...

machine-learning nlp data-analysis cosine-distance

asked Jul 22 '19 at 14:35

slowmonk

4735 silver badges13 bronze badges

3

votes

1answer

4k views

memory error in matrix cosine_similarity

I have (20905040, 7) of a dataset to recommend 10 different product to the user it could be larger than that but anyway I got memory error when processing the ...

scikit-learn recommender-system cosine-distance movielens

asked Jul 21 '19 at 11:35

jake Monk

491 silver badge3 bronze badges

1

vote

1answer

469 views

Elbow method for cosine distance

I have clustered vectors by cosine distance using nltk clusterer. If I understand correctly, Y axis for elbow method in euclidian distance would be the sum of every distance (squared) between centroid ...

nltk cosine-distance

asked Jun 23 '19 at 11:43

Ruuza

112 bronze badges

0

votes

1answer

41 views

Document matching with more priority to certain features than others

I am working on recommendation systems wherein I need to match the similarity of 2 users. Now, I know that I can use Tfidf vectorizer to calculate the the cosine similarity score between them. But, ...

recommender-system cosine-distance

asked Jun 20 '19 at 12:23

Reckoner

1135 bronze badges

4

votes

1answer

4k views

Calculating cosine similarity between 3D arrays using Python

I have two matrices with multiple columns and three rows each. I calculated the cosine similarity (sklearn) but it gives the result as a matrix. How can I obtain one single value? The matrices are the ...

python cosine-distance bert matrix

asked Jun 18 '19 at 10:36

GAYATRI VENUGOPAL

411 silver badge2 bronze badges

1

vote

1answer

59 views

Checking TF-IDF Results

I am working with TF-IDF and cosine similarity to do document comparisons and given a document, which document in the data is the most similar. However, sometimes it returns a high similarity between ...

tfidf cosine-distance similar-documents

asked Jun 16 '19 at 13:01

user1111111111

111 bronze badge

4

votes

1answer

910 views

word2vec word embeddings creates very distant vectors, closest cosine similarity is still very far, only 0.7

I started using gensim's FastText to create word embeddings on a large corpus of a specialized domain (after finding that existing open source embeddings are not performing well on this domain), ...

word2vec word-embeddings gensim embeddings cosine-distance

asked May 31 '19 at 10:35

Oren Matar

2211 silver badge6 bronze badges

1

vote

1answer

6k views

Hierarchical clustering with precomputed cosine similarity matrix using scikit learn produces error

We want to use cosine similarity with hierarchical clustering and we have cosine similarities already calculated. In the sklearn.cluster.AgglomerativeClustering documentation it says: A distance ...

python scikit-learn clustering cosine-distance

asked May 14 '19 at 20:21

M. Kaan

131 silver badge3 bronze badges

2

votes

3answers

2k views

Cosine similarity with arrays contaning NaN

I am trying to calculate a cosine similarity using Python in order to find similar users basing on ratings they have given to movies. As it can be expected there are a lot of NaN values. I am using ...

python recommender-system numpy cosine-distance

asked Apr 27 '19 at 13:15

user641597

1031 silver badge7 bronze badges

0

votes

2answers

4k views

how to create word2vec for phrases and then calculate cosine similarity

I have just started using word2vec and I have no idea how to create vectors (using word2vec) of two lists, each containing set of words and phrases and then how to calculate cosine similarity between ...

python word2vec data-analysis cosine-distance

asked Apr 5 '19 at 7:49

user3778289

1291 silver badge4 bronze badges

2

votes

3answers

109 views

Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...

similarity distance text cosine-distance

asked Feb 28 '19 at 6:55

Kavita Aherwar

314 bronze badges

2

votes

1answer

40 views

When I would use a specific similarity coefficient over another?

Like using Jaccards over Dice. I want real examples, of when I would prefer to use Jaccards, Dice, Cosine or any other similarity coefficient.

similarity cosine-distance jaccard-coefficient

asked Feb 3 '19 at 12:10

cantyousee

213 bronze badges

2

votes

1answer

2k views

cosine similarity between items (purchase data) and normalisation

I'm using IndexedRowMatrix which represents the products's user purchase behaviours and in order to build product recommendations, I use cosine similarity to calculate similarities between products. ...

recommender-system apache-spark pyspark cosine-distance

asked Nov 19 '18 at 10:20

SarahData

1711 silver badge6 bronze badges

1

vote

0answers

34 views

Best way to find dissimilarity in a 6x2 DataFrame?

I'm new to data science and am currently learning different techniques that I can do with Python. Currently, I'm trying it out with Spotify's API for my own playlists. The goal is to find the most ...

similarity dataframe k-nn cosine-distance

asked Nov 6 '18 at 4:14

Mustafa

1111 bronze badge

1

vote

1answer

33 views

Can I sum up feature vectors of a user‘s collection?

I want to find items that are similar to items users already have in their collection. Every item has attributes, so I created feature vectors where every element of the vector represents an attribute ...

recommender-system similarity cosine-distance vector-space-models

asked Sep 14 '18 at 21:33

Lurca

1111 bronze badge

3

votes

3answers

802 views

clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...

clustering k-means similarity distance cosine-distance

asked Jul 9 '18 at 13:50

jakes

8510 bronze badges

2

votes

0answers

1k views

Clustering with cosine similarity with specific threshold (in Python)

Given : 1, either given the origin data matrix X , such as X = [[1,1,1,1],[1,1,-1,-1],[1,1,1,-1]] or given the cosine similarity matrix of the original data X , ...

python numpy cosine-distance scipy

asked May 4 '18 at 19:24

Bill Ancalagon the black

1213 bronze badges

4

votes

1answer

4k views

How to find similar time series?

I've got a collection of yearly data (one value per year per category), and I'd like to find series that are most similar to one another. Example data is here. I don't know much about data science, ...

time-series similarity cosine-distance

asked Mar 19 '18 at 20:16

user2315852

711 silver badge4 bronze badges

Questions tagged [cosine-distance]

Related Tags