Questions tagged [similarity]

The tag has no usage guidance.

66 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
4
votes
2answers
51 views

Cluster elements that appear in the same lists

Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together. For example Given the following sets: ...
2
votes
1answer
89 views

How to build a symmetric similarity model on top of embeddings?

I have two equal length vectors that come out of two identical embedding layers. I want to calculate their similarity, and I don't trust the embedding layer enough to just use dot product (e.g. it's ...
2
votes
0answers
115 views

Similarity Measure Time Series

In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
2
votes
0answers
30 views

Measure of variety within list/cluster

I have a dataset of about 53000 points. It has been clustered twice, based on two sets of unrelated attributes. For the first clustering (clustering 1) I used DBScan, and it ended up with about 700 ...
2
votes
1answer
68 views

Similar students using Machine Learning

I have a student performance data, where I have marks of various subjects for the students and I want to find similar students with good marks in a particular subject using machine learning. How do I ...
2
votes
1answer
33 views

Identifying documents similar to specific clusters

Through performing clustering on a set of 1,000,000 text documents, I have identified 100 clusters. I am particularly interested in, say, 10 of the clusters. Imagine, I now have an additional set of ...
2
votes
3answers
428 views

clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
2
votes
0answers
644 views

Is there an algorithm or NN to match two documents, basically not closely similar?

Is there an algorithm or NN to match two documents? One is a claim description (e.g. a CV or product offer) and another is a requirements description (e.g. vacancy description or RFP). They are not ...
2
votes
0answers
308 views

Recommendation matrix as a product of User Similarity and Ratings

For both item-item and user-user collaborative filtering the recommendation matrix $Γ_{m x n}$, which is an (m x n) matrix, can be defined as: $$Γ(i,j)=r_{ij}$$ For user-user collaborative ...
2
votes
0answers
90 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf https://chrisjmccormick.wordpress.com/...
2
votes
0answers
463 views

How to fix similarity matrix in Pandas returning all NaNs?

I've been trying to create a similarity matrix in Pandas from with a matrix multiplication operation on a document-term count matrix with 2264 rows and 20475 columns. The calculation completes in ...
1
vote
0answers
20 views

Options to find the most similar question in a dataset of question-answer pairs?

I am building a chatbot that will only handle FAQs, but these FAQs are very specific to an organisation, so I cannot use any existing off-the-shelf solutions, or connect to question-answering APIs. I ...
1
vote
1answer
31 views

Calculate Similarity using User's Personal Data?

I want to find out which users are similar to each other using their personal/organisational data, such as department, company, site, etc. I have this data in a boolean format, as shown below: ...
1
vote
0answers
15 views

document similarity when the document size less than 30 tokens?

I was solving a problem to compare 3 million, 2018 documents against the 2019 documents. There are three text attributes to be compared from one item against the other. I used Latent Semantic Indexing ...
1
vote
2answers
78 views

How can we perform STS(Semantic Textual Similarity) on UnSupervised dataset using Deep Learning?

How to implement STS(Semantic Textual Similarity) on unlabelled dataset. Dataset column contains Unique_id, text1(contains paragraph), text2(contains paragraph). Ex: Column representation: Unique_id ...
1
vote
2answers
48 views

Similarity Measure of Simulated Time Series vs Observed time Series

In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
1
vote
0answers
13 views

Looking for an algorithm that compute similarity between a phrase and possible combination of tokens

I want to find similarity between a phrase and possible combination of tokens that may form the phrase. For example, phrase = 'sea surface water' Possible token = ['sea','surface land', 'surface ...
1
vote
0answers
41 views

Measuring the similarity between a numeric data matrix and one or more categorical variables?

Given a numeric data matrix $A$ of size $n \times p$, which each row represents an observation along $p$ variables, and a second categorical data matrix $M$ of size $n \times z$, where each row ...
1
vote
1answer
35 views

how to match a sentence to a cluster of keywords?

I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below. <...
1
vote
0answers
227 views

similarity measures for categorical data in data mining

I am working on my assignment in which i have to mention 5 similarity measures for categorical and continuous data in data mining. As a beginner I tried my best and found SQUARE DISTANCE,EUCLIDEAN AND ...
1
vote
0answers
264 views

Doc2vec most similar document to a query string

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title ...
1
vote
0answers
43 views

Clustering and graphing similarities of sentence subjects

I have a bunch of sentences. Each sentence is given a weight of how "close" it is to a particular subject. Ex. "I love reading math books" Subjects for the above sentence = ...
1
vote
0answers
78 views

python - Identify variable in similar sentences

I'm looking to solve the following problem: I have a list of similar sentences as my dataset, and I want to be able to type a new sentence, which is also similar to the sentences in my dataset and ...
1
vote
0answers
59 views

How to check the similarity between two transition matrixes

I have two transition matrixes in which the probabilities for the transition between each from state to each to states are ...
1
vote
0answers
25 views

Best way to find dissimilarity in a 6x2 DataFrame?

I'm new to data science and am currently learning different techniques that I can do with Python. Currently, I'm trying it out with Spotify's API for my own playlists. The goal is to find the most ...
1
vote
1answer
44 views

Recommender system that connect users with each other , should I go for content based or collaborative filtering?

I am trying to build a system where user come on the platform and he chooses a topic(predefined few topics) and then we connect him with any random online user who chooses the same topic. Then they ...
1
vote
0answers
23 views

Can I sum up feature vectors of a user‘s collection?

I want to find items that are similar to items users already have in their collection. Every item has attributes, so I created feature vectors where every element of the vector represents an attribute ...
1
vote
0answers
80 views

Counting similarity between two GIT commits

I am writing as part of my master thesis work an analysis of developers' work. I am looking for a new parameters to be consumed in Topological Data Analysis process. I would like to compare a ...
1
vote
0answers
132 views

Siamese Network Triplet Loss for unlabeled images

Hi I am rather new to machine learning so I'm not really sure whether it's possible or not. Right now I'm trying to train a siamese network to identify whether an image is the same or not(by one-shot ...
1
vote
0answers
470 views

Deep Learning - Find most similar images - Triplets vs Pairs

I am working with Python, scikit-learn, keras and with 450x540 rgb images of front-faced watches (e.g. Watch_1, Watch_2). My aim to run an autoencoder or a Siemese Neural Network to find the most ...
1
vote
0answers
262 views

Image similarity: Similarity of mixed vector

In order to identify the similarity between images (products) I want to use a neural network approach similar to TiefVision. This pre-trained neural network is basically translating the images into a ...
1
vote
0answers
175 views

Finding lookalike for large number of users

We have a large user-base, within which we want to find lookalikes, around 25-30 Million, for the users, we have data such as show liked by the user, genre liked etc. What we need to do is ...
1
vote
0answers
47 views

Analyzing Web page structure

I'm looking for an algorithm, that classifies a webpage. But not the content. I know the readability algorithm (by arc90) to extract a main text from an webpage. That for instance can decide, wether a ...
1
vote
0answers
335 views

Is there a parallel to record linkage/entity resolution where ML can be applied to the data for schema matching?

In the context of collecting disparate data sets holding similar information, are their examples of algorithms being able to resolve attributes of records being similar (while their values are ...
1
vote
0answers
77 views

What is the best way for cluster sentences using information provided by a POS tagger?

I have a small set of sentences (around 20) and I want to cluster them. Only features I have from every sentence is the output of a POS tagger that I apply to them before. How could I approach this ...
1
vote
0answers
507 views

What is the state of the art method for synonym detection?

In Natural Language Processing and Computational Linguistic what methods are deemed as SOA for similar word extraction . Can anyone direct me to those resources?
1
vote
0answers
41 views

Subgraph isomorphism and Anti-monotone property

While finding frequent subgraphs in single large graph, subgraph isomorphism (test) is not considered because its not anti-monotone. How and why subgraph isomorphism is not anti-monotone ?
0
votes
1answer
32 views

how to build word2vec content based recommendation?

I am building a content-based recommendation system for hotel accommodation. I have a hotel name, hotel description and location. I combined hotel name, description and location. Then, applied NLP and ...
0
votes
0answers
14 views

Is there a way to normalize a similarity matrix by row and column in a way such that only one entry per row or column is approximately 1

I am computing similarities between 2 vectors. My goal is to have approximately 1 matching sample with similarity ~1, for each sample, without having any samples that are similar to many other samples....
0
votes
0answers
42 views

Semantic Search Help

There is a problem we are trying to solve where we want to do semantic search on our set of data, i.e we have a domain specific data (example: sentences talking about automobiles) Our data is just a ...
0
votes
0answers
13 views

How to get a percentage of similarities of new data added based on model trained using a huge training set in machine learning

I need to find similarities in newly added data using machine learning. We have a huge data set with more than 200 000 rows having following information: Name D.O.B City (From predefined list) Phone ...
0
votes
0answers
8 views

How to draw a support set when classifying using Siamese networks without performing one shot learning?

How to perform classification on a test set with Siamese networks when I cannot afford to draw the support set from the test set itself? Possible options which come to my mind are: KNN using samples ...
0
votes
2answers
262 views

Question about Similarity vs Dissimilarity Matrix

Right now, I'm working on a coming up with a similarity vs dissimilarity matrix for a set of data points for a clustering algorithm. My question is, if I want to use one of the many clustering ...
0
votes
0answers
5 views

How to identify similar datasets based on multiple temporal variables?

I have a dataset that describes movie releases, i.e. day by day number of seats allocated to the movie and number of seats sold at different geographic locations. This dataset looks something like ...
0
votes
0answers
13 views

Matching resumes with jobs in a NoSQL database

I have a series of user resumes and would like to find the best matching jobs per resume, the task seems simple at first except I have a nosql db holding the jobs (1m+), my initial idea was to add a ...
0
votes
0answers
55 views

Modifying BERT sentence encodings

I'm using BERT to encode sentences. The sentences I'm encoding are quite similar, meaning they all belong to the same overall topic. Therefor, I am using another parameter for measuring similarity. ...
0
votes
0answers
12 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
0answers
28 views

Is it necessary to perform Z-score or Min-Max normalization on L1-normalized data?

I have a dataset which contains vectors that generated from subtitles and have been L1 normalised, I want to calculate cosine similarity & Euclidean distance, I thought it is better if I use Z-...
0
votes
0answers
15 views

supervised similarity scoring system - recommender system

I have a dataset of movie collections with 10-15 features describing each movie. I also have a dataset of user ratings of the similarity between some random pairs of movies. Using both of these data ...
0
votes
0answers
25 views

What is the meaning of constraint two neural networks against each other?

I have two identical autoencoders with the same number of layers and parameters. I would like to find the similarity between two images from different domains such as an image captured from the camera ...