The similarity tag has no usage guidance.
0
votes
1answer
15 views
Appropriate similarity measure that highlights symmetrical values of features
I am trying to find an appropriate distance measure that reflects the differences of the vectors seen in the image below:
The green vector is compared with the blue one and the orange one.
Most of ...
0
votes
0answers
10 views
Implementing triplet loss for an arbitrarily large number of class
I'm trying to implement a triplet loss for a large number of class. I've checked out a few of its implementations in PyTorch. However, they are usually used for datasets with a limited number of ...
2
votes
0answers
8 views
Siamese Network Triplet Loss for unlabeled images
Hi I am rather new to machine learning so I'm not really sure whether it's possible or not. Right now I'm trying to train a siamese network to identify whether an image is the same or not(by one-shot ...
0
votes
1answer
46 views
Text Similarities: which nlp methods to use?
I have data where there is text for each user A visiting a business B. I want to find similarity between each user using their text.
Question 1: Which NLP method should I start with?
I have tried ...
1
vote
2answers
26 views
Finding best cluster after running K-Means clustering
I have bunch of text which I want to segregate based on semantic similarity. Running through K-Means, I was able to divide the complete text into different clusters.
However, I still need to find ...
0
votes
1answer
20 views
Can cosine similarity be applied to multidimensional matrices?
I'm trying to find the similarity between two 4D matrices. Because cosine similarity takes the dot product of the input matrices, the result is inevitably a matrix. Is there a way to get a scalar ...
1
vote
1answer
25 views
Clustering 1-gram Strings
I have a big list of mail addresses (around one million) and I want to use/adapt/create an algorithm to find similarities between them (basically to cluster them).
All the algorithms I've checked so ...
2
votes
1answer
22 views
Finding the most phonetically similar word from WordNet
Besides soundex and other libraries, that take two words and determine whether they are similar, is there any way to find the most similar sounding word from WordNet, for a given word? I tried to use ...
1
vote
2answers
53 views
clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure
I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
0
votes
0answers
27 views
Estimate the number of very distinct elements inside a dataset
Lets take for example the MNIST dataset, this dataset is composed of 60 000 training samples each sample is a handwritten digit between 0 and 9. however, we may find several samples inside this ...
0
votes
1answer
18 views
Voting patterns similarities
I'm interested in any research materials on voting patterns.
I have a data set of how PMs (members of parliament) voted in my country during last couple of years. Each PM has 3 buttons: Yes, No, ...
1
vote
2answers
29 views
Grouping company information
I have 3 different datasets with company information, in all of them I have company name, but is not perfect: For example:
Dataset A: Company name: Facebook
Dataset B: Company name: Facebook, Inc
...
3
votes
1answer
75 views
Are there any very good APIs for matching similar images?
I am working with Python. I have 3000 thousands images of front-faced watches like the following ones: Watch1, Watch2, Watch3.
I want to find an API which receives this collection of photos or even ...
2
votes
1answer
65 views
How can I measure the similarity between 2 IP addresses? Is there any code to re-use?
I need to measure the similarity of 2 IP addresses. I could not find any sample code in scala or other languages to find the distance between 2 IPs.
3
votes
2answers
110 views
Are there any good NLP APIs for comparing strings in terms of semantic similarity?
I want to create a chatbot which informs the user about traffic at the streets but not in real-time for the moment. I have created a small database with MySQL which has some data stored regarding ...
0
votes
1answer
45 views
Image Matching to solve captcha
I am building a bot with python and I need some system to solve captchas like these:
I think I need a deep learning algorithm, but coding one is a pain in the ass. Is there any easy solution to this?
...
1
vote
2answers
75 views
intra-cluster similarity metric
I have some observations belonging to groups, and I want to compute the similarity of those observations within the different groups in order to tell that observations within specific group have ...
3
votes
1answer
538 views
Unsupervised Anomaly Detection in Images
I would like to detect defects/anomalies in images. Due to the lack of images with anomalies, I try to solve the problem in an unsupervised manner. Until now, I trained a variational autoencoder ...
0
votes
1answer
163 views
How to find similar time series?
I've got a collection of yearly data (one value per year per category), and I'd like to find series that are most similar to one another.
Example data is here.
I don't know much about data science, ...
0
votes
0answers
70 views
User-user suggestions with collaborative filtering
I have a binary matrix N x N where both rows and columns represent users of a website. If matix[i,j] = 1 it means that ...
1
vote
0answers
261 views
Deep Learning - Find most similar images - Triplets vs Pairs
I am working with Python, scikit-learn, keras and with 450x540 rgb images of front-faced watches (e.g. Watch_1, Watch_2).
My aim to run an autoencoder or a Siemese Neural Network to find the most ...
0
votes
4answers
122 views
Similarity metric for clustering
I am working on a clustering problem. I am not able to find the right similarity metric for my system.
I have n nodes with ordered vector (eg: [1,0,0,0,1,0,.....] "1" represents represent presence of ...
3
votes
2answers
61 views
Collection Of Variable Length Sequences and Descriptions: A Search Problem
I have a tough problem and need some advice:
Suppose I have a collection of variable length sequences, many of which are unique -- imagine the moves to a chess game, eg
d4 Nf6
c4 g6
Nc3 Bg7
...
1
vote
1answer
149 views
Which dissimilarity/similarity measure use after a dimension reduction ( PCA / AutoEncoder / … )?
Each problem required its own similarity/dissimilarity measure. Imagine we are dealing with dataset composed with vector of real. I suppose that we mostly use the euclidean distance especially in low ...
1
vote
2answers
68 views
How to validate different method of clustering
I have applied K-means and Hierarchical Agglomerative clustering method on some data and clustered them into 5 groups. For validation (agreement) purpose I used this formula:
...
0
votes
0answers
49 views
How to use ALS for similarity model?
I have implemented a similarity model using item-item collaborative filtering. The predicted rating function in my case is:
$$r\hat()=\mu + b_{u}+b_{i}+Sim_{i,j}(r_{u,i})$$
In order to minimize the ...
1
vote
0answers
107 views
Image similarity: Similarity of mixed vector
In order to identify the similarity between images (products) I want to use a neural network approach similar to TiefVision. This pre-trained neural network is basically translating the images into a ...
0
votes
0answers
114 views
Clarity about sparse matrices in recommendation systems
When we are building a recommendation system and have a very sparse matrix, we use model based collaborative filtering.
Is there any specific reason as to why we don't use
...
4
votes
1answer
247 views
How do you set sigma for the Gaussian similarity kernel?
Let's say we have $n$ two-dimensional vectors: $$\mathbf{x}_1,\dots,\mathbf{x}_i,\dots,\mathbf{x}_n=(x_{1_1},x_{1_2})^T,\dots,(x_{i_1},x_{i_2})^T,\dots,(x_{n_1},x_{n_2})^T$$ How do you set $\sigma$ ...
0
votes
1answer
107 views
How to find a similarity value between cars
So I have a database of web-scraped cars, and I want to find the similarity between cars based on the km driven (e.g 69000), the model year (e.g 2012), and the trim of the car which will be one of ...
1
vote
1answer
156 views
Calculate similarity on boolean data
I am trying to implement simple recommender system and I am trying to understand different approaches to achieve my goal.
My dataset consists of users and items that they bought. I have information ...
0
votes
1answer
72 views
What is the efficient way to generate a similarity score when comparing two face images?
I am working on a face recognition application using deep learning. To plot the ROC curves and do performance evaluation, I extracted the features from the last layer of the deep neural network and I ...
5
votes
2answers
7k views
Best practical algorithm for sentence similarity
I have two sentences, S1 and S2, both which have a word count (usually) below 15.
What are the most practically useful and successful (machine learning) algorithms, which are possibly easy to ...
0
votes
0answers
28 views
Search for LSH similar buckets
I am using minhash to find similar vectors in a matrix. I am using LSH to band minhash values. Now I would like to find items with 2 or more similar bands between them.
What is the most efficient way ...
0
votes
2answers
765 views
Finding similarity between two histogram plots
I have data census(name, sex, age, capital_gain) and I want to plot all possible views (in a histogram) and find the most interesting view based on the distinction to other views.
So, I have to ...
-1
votes
2answers
132 views
Assessing Group Similarities and Dissimilarities Post PCA
The goal is to assess similarity and dissimilarity between 6 known groups.
The original data began with the 6 known groups and 2,700+ variables all on a scale of 0 to 100.
I have performed PCA to ...
0
votes
0answers
28 views
How can you have a directed edit distance less than 1?
I am reading through Meme-tracking and the Dynamics of the News Cycle (Leskovec, Backstrom, Kleinberg) and came across a part that is confusing me.
Given a set of phrases, they wish to develop a ...
0
votes
1answer
196 views
Finding similar articles in realtime [closed]
I want to build a large document (news article) searchable database, such as when adding a new article I will be able to quickly find X most similar articles from it.
What is the right tech/algorithm/...
1
vote
0answers
675 views
Doc2vec to calculate cosine similarity - absolutely inaccurate
I'm trying to modify the Doc2vec tutorial to calculate cosine similarity and take Pandas dataframes instead of .txt documents. I ...
1
vote
1answer
1k views
Cosine similarity between query and document confusion
I am going through the Manning book for Information retrieval. Currently I am at the part about cosine similarity. One thing is not clear for me.
Let's say that I have the tf idf vectors for the ...
3
votes
4answers
7k views
Sentence similarity prediction
I'm looking to solve the following problem: I have a set of sentences as my dataset, and I want to be able to type a new sentence, and find the sentence that the new one is the most similar to in the ...
0
votes
1answer
776 views
Create similarity matrix
I have a training set and a testing set of vectors. All the vectors are labeled.
For each labeled vector in the testing set, there are 3 vectors in the training set with the same label.
I'm using ...
1
vote
1answer
241 views
Computing Item-to-Item Similarity Using Cosine
I have a "User x Item" matrix as below:
user item1 item2 item3
u1 2 0 3
u2 1 2 0
u3 4 3 1
u4 0 2 2
I want to computer the ...
1
vote
1answer
279 views
Choosing the right corpus to build a TF IDF Vectorizer used for comparing the similarity of two strings
I am working on evaluating when a pair of string objects can be considered equal (e.g. given that we are talking about journals, is "international journal of air and water pollution" the same of "air ...
2
votes
0answers
329 views
Is there an algorithm or NN to match two documents, basically not closely similar?
Is there an algorithm or NN to match two documents? One is a claim description (e.g. a CV or product offer) and another is a requirements description (e.g. vacancy description or RFP). They are not ...
1
vote
2answers
302 views
What techniques should I use to compare the similarity between a bunch of texts?
If I have a list of job postings stored as raw texts and I want to compare the similarity of all the job postings to a given resume, what techniques or algorithms should I use?
I'm thinking of ...
0
votes
1answer
125 views
How do I calculate a similarity matrix with a Student-t kernel?
As the title says, how do I calculate a similarity matrix with an un-normalized Student-t kernel? I'm attempting to calculate Kullback-Leibler divergence for different t-SNE runs, but need a Q-matrix ...
1
vote
0answers
129 views
Finding lookalike for large number of users
We have a large user-base, within which we want to find lookalikes, around 25-30 Million, for the users, we have data such as show liked by the user, genre liked etc.
What we need to do is ...
1
vote
1answer
212 views
Word2vec using gensim
I am using gensim library to find most similar words to some words that I have. Using 10000 data samples (short text mainly 1-2 sentences) to train, I get really bad results! Why this is so? Also by ...
1
vote
0answers
39 views
Analyzing Web page structure
I'm looking for an algorithm, that classifies a webpage. But not the content. I know the readability algorithm (by arc90) to extract a main text from an webpage. That for instance can decide, wether a ...
