Questions tagged [similarity]
The similarity tag has no usage guidance.
231
questions
0
votes
0answers
12 views
Can a siamese neural network distinguish expected from unexpected changes?
Please redirect me to another stack exchange if this isn't the appropriate forum. I am interested in finding a neural network architecture that can detect distance between two inputs. As an example, ...
0
votes
1answer
24 views
best similarity measure for images with different angles
I want to compare different images (where the images are of the same setup but the angles with which the images are taken are different). I want to obtain some sort of similarity score. I tried using ...
0
votes
1answer
19 views
Non-commutative distance formula
I am trying to find a distance formula or a method that can give the non-commutative distance between two points in a feature space.
Suppose there are two movies represented in an R^n feature space. ...
1
vote
1answer
48 views
Similarity between 2 statistical distributions
Is there any index that measures similarity between 2 gaussian distributions of 1-D data (may have slightly different number of points) considering their mean shift, variance shift, difference in ...
2
votes
2answers
33 views
Find 10 most similar data points from 100?
I am trying to solve a problem where I need to group battery cells that are most similar to each other to form a reliable battery pack. It seems like a clustering problem but I only need to find the ...
0
votes
0answers
13 views
How does LDA model do inference on new documents?
Through Latent Dirichlet Allocation (LDA) one can learn topics from texts. The trained topic model can then be used on unseen data to infer to what extent each of the learned topics are represented in ...
1
vote
1answer
51 views
Why does an imbalanced data set badly effect distance measures like Mahalanobis?
I'm relatively new to data science and I am struggling to understand why the Mahalanobis distance (or any other distance measure) applied to an imbalanced data-set becomes inaccurate. I have a data ...
0
votes
1answer
14 views
Siamese vs matching network for correct image category matching
I have to find the closest match between my image and bunch of already collected images of different classes in the folder. Whic meta-learning approach should I select. I am thinking about the Siamese ...
1
vote
1answer
53 views
Hierarchical clustering with the consensus matrix as similarity matrix
I'm following this article on consensus clustering in Python programming. On page 7 the authors state that "The consensus matrix lends itself naturally to be used as a visualization tool to help ...
0
votes
0answers
13 views
How to find the most similar image between two datasets?
How do you measure the image similarity using (Mean squared error), find the most similar one, and save them as a pair of images in a folder with their similarity.
I have two images datasets:
...
0
votes
1answer
15 views
Similarity Matching Algorithm
I am looking for help on identifying a class of algorithm. If I have a tabular training and test set I want to know the similarity of rows based on some numeric features. The training data would be ...
0
votes
0answers
14 views
Is there a way to calculate the cosine distance between 2 time series?
Let's say I have time series data of City A, City B, City C & City D that looks like this:
...
1
vote
1answer
28 views
How to find coherence between a large number of sentences
I have a list of sentences returned as a result of a document search algorithm. I want to determine if the results returned are semantically close/similar/coherent using some sort of metric. For a ...
3
votes
3answers
137 views
Dot product for similarity in word to vector computation in NLP
In NLP while computing word to vector we try to maximize log(P(o|c)). Where P(o|c) is probability that o is outside word, given that c is center word.
Uo is word vector for outside word
Vc is word ...
1
vote
1answer
31 views
group similar subjects and train only using them
I have a dataset with 5k subjects. It's a binary classification problem where I have 3000 positive and 2000 negative subjects.
Now to build a model, I don't like to train the usual way (where we build ...
0
votes
1answer
46 views
Calculate the similarity between pairs of time series data
I have 5 pieces of time series data. It is the weekly sales of 5 different stores (A,B,C,D,E). There are no missing values. A quick visual inspection shows that these 5 pieces of time series data have ...
1
vote
0answers
22 views
Scoring metric for recommendation system
I'm working on a project that involves building a news recommendation system. I've come as far as quantifying user interaction with different articles on the site into user's affinity towards atopic ...
3
votes
1answer
118 views
Pearson correlation with data sets that have values on different scales [closed]
I have two datasets with which I want to do a Pearson correlation analysis. I have carried out the analysis which makes sense, however I want to be sure it is valid given that both datasets have ...
1
vote
1answer
14 views
Similarity Threshold Standards
When using similarity measures (eg. Resnik Information Content, Cosine Similarity, etc.) for any type of data, are there any standard similarity thresholds that are used, or does it all depend on the ...
0
votes
0answers
14 views
Similarity between two sets of documents
I have 4000 tweets about a brand and 2000 comments about its products. I want to know how much the comments about the brand's products are related to the tweets it generates.
I know how to do a cosine ...
0
votes
1answer
30 views
Similarity between two time series with different sampling frequency, different amplitude, and different lengths but taken from the same source?
I have two files with accelerator readings and I want to get some metric/ measurement to get the similarity between these two files. I have tried Pearson’s R coefficient, dtw distance, dtw score. ...
1
vote
1answer
37 views
Recommender System for mostly unique user and items
I am trying to develop a recommender system for a job matching problem. My data consists of past matched candidate profiles and job profiles as well as if there was a success such that both, candidate ...
0
votes
0answers
8 views
Appearance-based hashing for similarity detection for picking the 100 most distinct images out 500 images
I would like to perform appearance-based hashing for similarity detection. I have 500 photos for each of my categories but I only want to maintain the 100 of them that are most distinct. How should I ...
2
votes
1answer
28 views
What are the Most Dissimilar MNIST Digits?
Using whatever definition of dissimilarity over sets that you'd like, what are the most dissimilar two digits in MNIST? I was thinking that a reasonable approach to answering the question would be to ...
2
votes
1answer
25 views
How is an ASR's output compared to ground truth for validation?
I am curious how it is done as I am interested in doing something similar. I have some manually transcribed data that contains tags for multiple speakers. I want to compare how well the out of the box ...
3
votes
1answer
159 views
What is the correct formula for Jaccard coefficient with integer vectors?
I understand the Jaccard index is the number of elements in common divided by the total number of distinct elements. But it seems to be some discrepancy or terminology confusion about Jaccard being ...
0
votes
0answers
18 views
product recommendation of a single product based on customer similarity?
I have been wondering how you can build a model to recommend only one single product to a bunch of customers. So basically the question that I would like to answer with this model is to have a ranking ...
1
vote
1answer
20 views
Fuzzy and FuzzyWuzzy: what are the differences in text comparison?
I have found a lot of information about fuzzy logic, but less information about fuzzywuzzy. I would like to know more about this, the function which determines the logic, if possible, and understand ...
1
vote
0answers
24 views
Is the magnitude of a word vector correlated with the frequency of the word in a text?
In order to find the similarity between words, cosine similarity seems to be the most common measure to use. In a conversation that I had about this topic, someone mentioned that words that mean more ...
1
vote
0answers
11 views
Which string distance equation for fuzzy-matching person names is reliable?
A reproducible example with a small bit of R code is available in this stackoverflow post (link so I dont need to re-type out the code). The fuzzytext library in R ...
0
votes
0answers
9 views
Link Recommendation System
I am trying to build a Recommendation system on my website for recommending similar articles to the user. Eg: Lets say a user is reading an article about sports on a news website. The next article ...
-1
votes
1answer
33 views
Equivalence of two separate data distribution
I am working in generative modelling. I have a real dataset of which I want to know the underlying distribution. For this I create a synthetic data generator which tries to mimic the real data. I ...
1
vote
2answers
79 views
State-of-the-art Python packages that can evaluate language similarity
I am trying to evaluate the likelihood of generating a specific sentence out of a large set of sentences. To do this, I start from a simple approach: training a custom n-gram language model and ...
0
votes
1answer
23 views
Modelling for similarity between two descriptions
I have a dataset of companies and research projects that they were involved in. A subset of the dataset is shown below.
I am trying to find a way to model similarity between the company and the ...
0
votes
0answers
36 views
Why do all DTW-packages use different step-patterns and distance-methods?
I already posted it on StackOverflow. I was not sure if this was the right place.
I have several questions about the algorithm DTW. I tried different Python-Packages
DTAI-Distance
DTW-Python
First ...
3
votes
2answers
1k views
Similarity Measure between two feature vectors
I have face identification system with following details:
VGG16 model for feature extraction
512 dimensional feature vector (...
0
votes
1answer
23 views
How to calculate similarity between 2 users based on the images they share?
Say there are 2 users, A and B, and they each shared 10 images (in some social media site), which I have collected in 2 folders separately.
I want to calculate the similarity between the 2 users based ...
1
vote
0answers
26 views
Pairwise 3D object correlation between 2 objects
I have a dataset which contains 3D CT scans from different patients along with the segmenation masks of a certain organ. The 3D scans have been drawn each day for a period of 30 days for each patient. ...
0
votes
0answers
9 views
Retrieving information from 2 or more approaches
I'm trying to extract information from documents. There are two approaches currently which produces the following cases where both approaches
...
1
vote
1answer
131 views
How to figure out if two sentences have the same meaning with AI?
I have two sentences which might be similar in meaning.
Are there a useful and successful (machine learning) algorithms, which is able to determine the semantic similarity?
Are there any approaches ...
0
votes
1answer
207 views
Is there a loss function that measures the cross similarity between two 2D tensors?
Given two input tensors x1 and x2 with the shape [batch_size, hidden_size], let ...
4
votes
1answer
306 views
Text classification based on n-grams and similarity
I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
0
votes
2answers
20 views
Scalable way to group users with similar titles purchased
I'm trying to figure out the best way to group customers based on checkout items in their shopping cart. I have the basket, and what's in the basket, but am at a complete loss on how to group all the ...
0
votes
2answers
38 views
Ideas to check if a given sentence A is the response to a query B
I've a sentence 'A' which is a possible response to one of the x number of questions. I need to compare the response with each sentence to see which question's response sentence 'A' is or not a ...
0
votes
1answer
45 views
Correlation between words, then texts
I have texts collected on different topics. I would like to study the possible correlation between these.
I have started looking at the word frequency and it seems that in one dataset the word with ...
0
votes
1answer
79 views
Text classification analysis based on similarity
I have been reading a lot of literature regarding text classification and different approaches/models, especially using Python language, but probably I am still missing something on how to build the ...
0
votes
1answer
227 views
Building a content-based music recommendation system
I am trying to build a recommendation system in Python that recommends songs based on a playlist.
What I have is two datasets:
1. One dataset consists of 350 songs from my playlist and 13 acoustic ...
0
votes
1answer
49 views
How to identify topic transition in consecutive sentences using Python?
I'm new to data mining. I want to detect topic transition among consecutive sentences. For instance, I have a paragraph (this could be a collection of dozens of sentences, sometimes without ...
1
vote
0answers
17 views
Scale (frequency) invariant cross-correlation
I am looking for cross-correlation of two 'signals' that would be invariant of scale(frequency, or sampling).
To give you example what I mean:
Standard cross-correlation of sin(x) and cos(x) find ...
1
vote
1answer
56 views
