Questions tagged [similarity]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
12 views

Can a siamese neural network distinguish expected from unexpected changes?

Please redirect me to another stack exchange if this isn't the appropriate forum. I am interested in finding a neural network architecture that can detect distance between two inputs. As an example, ...
0
votes
1answer
24 views

best similarity measure for images with different angles

I want to compare different images (where the images are of the same setup but the angles with which the images are taken are different). I want to obtain some sort of similarity score. I tried using ...
0
votes
1answer
19 views

Non-commutative distance formula

I am trying to find a distance formula or a method that can give the non-commutative distance between two points in a feature space. Suppose there are two movies represented in an R^n feature space. ...
1
vote
1answer
48 views

Similarity between 2 statistical distributions

Is there any index that measures similarity between 2 gaussian distributions of 1-D data (may have slightly different number of points) considering their mean shift, variance shift, difference in ...
2
votes
2answers
33 views

Find 10 most similar data points from 100?

I am trying to solve a problem where I need to group battery cells that are most similar to each other to form a reliable battery pack. It seems like a clustering problem but I only need to find the ...
0
votes
0answers
13 views

How does LDA model do inference on new documents?

Through Latent Dirichlet Allocation (LDA) one can learn topics from texts. The trained topic model can then be used on unseen data to infer to what extent each of the learned topics are represented in ...
1
vote
1answer
51 views

Why does an imbalanced data set badly effect distance measures like Mahalanobis?

I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data ...
0
votes
1answer
14 views

Siamese vs matching network for correct image category matching

I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese ...
1
vote
1answer
53 views

Hierarchical clustering with the consensus matrix as similarity matrix

I'm following this article on consensus clustering in Python programming. On page 7 the authors state that "The consensus matrix lends itself naturally to be used as a visualization tool to help ...
0
votes
0answers
13 views

How to find the most similar image between two datasets?

How do you measure the image similarity using (Mean squared error), find the most similar one, and save them as a pair of images in a folder with their similarity. I have two images datasets: ...
0
votes
1answer
15 views

Similarity Matching Algorithm

I am looking for help on identifying a class of algorithm. If I have a tabular training and test set I want to know the similarity of rows based on some numeric features. The training data would be ...
0
votes
0answers
14 views

Is there a way to calculate the cosine distance between 2 time series?

Let's say I have time series data of City A, City B, City C & City D that looks like this: ...
1
vote
1answer
28 views

How to find coherence between a large number of sentences

I have a list of sentences returned as a result of a document search algorithm. I want to determine if the results returned are semantically close/similar/coherent using some sort of metric. For a ...
3
votes
3answers
137 views

Dot product for similarity in word to vector computation in NLP

In NLP while computing word to vector we try to maximize log(P(o|c)). Where P(o|c) is probability that o is outside word, given that c is center word. Uo is word vector for outside word Vc is word ...
1
vote
1answer
31 views

group similar subjects and train only using them

I have a dataset with 5k subjects. It's a binary classification problem where I have 3000 positive and 2000 negative subjects. Now to build a model, I don't like to train the usual way (where we build ...
0
votes
1answer
46 views

Calculate the similarity between pairs of time series data

I have 5 pieces of time series data. It is the weekly sales of 5 different stores (A,B,C,D,E). There are no missing values. A quick visual inspection shows that these 5 pieces of time series data have ...
1
vote
0answers
22 views

Scoring metric for recommendation system

I'm working on a project that involves building a news recommendation system. I've come as far as quantifying user interaction with different articles on the site into user's affinity towards atopic ...
3
votes
1answer
118 views

Pearson correlation with data sets that have values on different scales [closed]

I have two datasets with which I want to do a Pearson correlation analysis. I have carried out the analysis which makes sense, however I want to be sure it is valid given that both datasets have ...
1
vote
1answer
14 views

Similarity Threshold Standards

When using similarity measures (eg. Resnik Information Content, Cosine Similarity, etc.) for any type of data, are there any standard similarity thresholds that are used, or does it all depend on the ...
0
votes
0answers
14 views

Similarity between two sets of documents

I have 4000 tweets about a brand and 2000 comments about its products. I want to know how much the comments about the brand's products are related to the tweets it generates. I know how to do a cosine ...
0
votes
1answer
30 views

Similarity between two time series with different sampling frequency, different amplitude, and different lengths but taken from the same source?

I have two files with accelerator readings and I want to get some metric/ measurement to get the similarity between these two files. I have tried Pearson’s R coefficient, dtw distance, dtw score. ...
1
vote
1answer
37 views

Recommender System for mostly unique user and items

I am trying to develop a recommender system for a job matching problem. My data consists of past matched candidate profiles and job profiles as well as if there was a success such that both, candidate ...
0
votes
0answers
8 views

Appearance-based hashing for similarity detection for picking the 100 most distinct images out 500 images

I would like to perform appearance-based hashing for similarity detection. I have 500 photos for each of my categories but I only want to maintain the 100 of them that are most distinct. How should I ...
2
votes
1answer
28 views

What are the Most Dissimilar MNIST Digits?

Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
2
votes
1answer
25 views

How is an ASR's output compared to ground truth for validation?

I am curious how it is done as I am interested in doing something similar. I have some manually transcribed data that contains tags for multiple speakers. I want to compare how well the out of the box ...
3
votes
1answer
159 views

What is the correct formula for Jaccard coefficient with integer vectors?

I understand the Jaccard index is the number of elements in common divided by the total number of distinct elements. But it seems to be some discrepancy or terminology confusion about Jaccard being ...
0
votes
0answers
18 views

product recommendation of a single product based on customer similarity?

I have been wondering how you can build a model to recommend only one single product to a bunch of customers. So basically the question that I would like to answer with this model is to have a ranking ...
1
vote
1answer
20 views

Fuzzy and FuzzyWuzzy: what are the differences in text comparison?

I have found a lot of information about fuzzy logic, but less information about fuzzywuzzy. I would like to know more about this, the function which determines the logic, if possible, and understand ...
1
vote
0answers
24 views

Is the magnitude of a word vector correlated with the frequency of the word in a text?

In order to find the similarity between words, cosine similarity seems to be the most common measure to use. In a conversation that I had about this topic, someone mentioned that words that mean more ...
1
vote
0answers
11 views

Which string distance equation for fuzzy-matching person names is reliable?

A reproducible example with a small bit of R code is available in this stackoverflow post (link so I dont need to re-type out the code). The fuzzytext library in R ...
0
votes
0answers
9 views

Link Recommendation System

I am trying to build a Recommendation system on my website for recommending similar articles to the user. Eg: Lets say a user is reading an article about sports on a news website. The next article ...
-1
votes
1answer
33 views

Equivalence of two separate data distribution

I am working in generative modelling. I have a real dataset of which I want to know the underlying distribution. For this I create a synthetic data generator which tries to mimic the real data. I ...
1
vote
2answers
79 views

State-of-the-art Python packages that can evaluate language similarity

I am trying to evaluate the likelihood of generating a specific sentence out of a large set of sentences. To do this, I start from a simple approach: training a custom n-gram language model and ...
0
votes
1answer
23 views

Modelling for similarity between two descriptions

I have a dataset of companies and research projects that they were involved in. A subset of the dataset is shown below. I am trying to find a way to model similarity between the company and the ...
0
votes
0answers
36 views

Why do all DTW-packages use different step-patterns and distance-methods?

I already posted it on StackOverflow. I was not sure if this was the right place. I have several questions about the algorithm DTW. I tried different Python-Packages DTAI-Distance DTW-Python First ...
3
votes
2answers
1k views

Similarity Measure between two feature vectors

I have face identification system with following details: VGG16 model for feature extraction 512 dimensional feature vector (...
0
votes
1answer
23 views

How to calculate similarity between 2 users based on the images they share?

Say there are 2 users, A and B, and they each shared 10 images (in some social media site), which I have collected in 2 folders separately. I want to calculate the similarity between the 2 users based ...
1
vote
0answers
26 views

Pairwise 3D object correlation between 2 objects

I have a dataset which contains 3D CT scans from different patients along with the segmenation masks of a certain organ. The 3D scans have been drawn each day for a period of 30 days for each patient. ...
0
votes
0answers
9 views

Retrieving information from 2 or more approaches

I'm trying to extract information from documents. There are two approaches currently which produces the following cases where both approaches ...
1
vote
1answer
131 views

How to figure out if two sentences have the same meaning with AI?

I have two sentences which might be similar in meaning. Are there a useful and successful (machine learning) algorithms, which is able to determine the semantic similarity? Are there any approaches ...
0
votes
1answer
207 views

Is there a loss function that measures the cross similarity between two 2D tensors?

Given two input tensors x1 and x2 with the shape [batch_size, hidden_size], let ...
4
votes
1answer
306 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
0
votes
2answers
20 views

Scalable way to group users with similar titles purchased

I'm trying to figure out the best way to group customers based on checkout items in their shopping cart. I have the basket, and what's in the basket, but am at a complete loss on how to group all the ...
0
votes
2answers
38 views

Ideas to check if a given sentence A is the response to a query B

I've a sentence 'A' which is a possible response to one of the x number of questions. I need to compare the response with each sentence to see which question's response sentence 'A' is or not a ...
0
votes
1answer
45 views

Correlation between words, then texts

I have texts collected on different topics. I would like to study the possible correlation between these. I have started looking at the word frequency and it seems that in one dataset the word with ...
0
votes
1answer
79 views

Text classification analysis based on similarity

I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
0
votes
1answer
227 views

Building a content-based music recommendation system

I am trying to build a recommendation system in Python that recommends songs based on a playlist. What I have is two datasets: 1. One dataset consists of 350 songs from my playlist and 13 acoustic ...
0
votes
1answer
49 views

How to identify topic transition in consecutive sentences using Python?

I'm new to data mining. I want to detect topic transition among consecutive sentences. For instance, I have a paragraph (this could be a collection of dozens of sentences, sometimes without ...
1
vote
0answers
17 views

Scale (frequency) invariant cross-correlation

I am looking for cross-correlation of two 'signals' that would be invariant of scale(frequency, or sampling). To give you example what I mean: Standard cross-correlation of sin(x) and cos(x) find ...
1
vote
1answer
56 views

Change distribution of a vector

I have the following vector ...

1
2 3 4 5