Questions tagged [similarity]
The similarity tag has no usage guidance.
0
votes
0answers
5 views
Any there databases with native support for applying a NN model to produce search rankings?
The Situation: I have a simple neural net with an input vector that consists of the euclidean distance between the attributes of two cars. (For example the attributes would include wheel size, car ...
0
votes
0answers
13 views
Semantic Similarity in Universal Sentence Encoder
I am currently using Universal Sentence Encoder to embed certain sentences which I would then feed to a deep learning model to do some prediction, but just to test whether the universal sentence ...
0
votes
0answers
12 views
Features Vectors in embedding space
I have a bunch of users, each of them with about 100 features.
My goal is to create an embedded space to compute the "distance" between users.
Also, I want to be able to visualize it with Tensorboard (...
2
votes
2answers
52 views
Similarity score: Can Sklearn SVR predict values greater than 1 and less than 0?
I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve a similarity problem. Using GridSearchCV, I am finding the best ...
4
votes
6answers
131 views
Distance between users
I want to compute the "distance" between users in order to return the top n similar users, for any given user.
For each user a have a bunch of features.
This is close to a recommendation system, ...
1
vote
1answer
19 views
identify similarities in a dataset
I have a dataset of customers:
age height weight eye_colour
30 174 74 Nan
20 191 71 Nan
28 165 56 Brown
...
I would like to ...
1
vote
1answer
21 views
how to match a sentence to a cluster of keywords?
I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below.
<...
1
vote
1answer
38 views
Jaccard similarity calculate similarity
It is not clear to me how to calculate similarity between two products from the example.
How do they calculate that?
1
vote
2answers
30 views
How can I measure if a population has the same distribution as other?
Is there a population similarity index of some kind which could help me determine if two populations in two different datasets are the same or at least similar?
The datasets have the exact same ...
3
votes
4answers
41 views
Best way to combine two similar document
I have f.ex.: two news-articles that report the same event. However, these two text are similar BUT not the same. I would like to combine these two texts creating one text that contains only the most "...
2
votes
0answers
68 views
Jaccard Similarity with Binary Data
I have 5400 rows of data and 3211 columns of attributes.
The first 4 columns are ID/Name/ParentID/ObjectType - the rest of the 3207 columns are the attributes that are to be used for similarity ...
1
vote
0answers
27 views
similarity measures for categorical data in data mining
I am working on my assignment in which i have to mention 5 similarity measures for categorical and continuous data in data mining.As a beginner i tried my best and found SQUARE DISTANCE,EUCLIDEAN AND ...
0
votes
0answers
9 views
how to get similary of unseen document with whole corpus
I have a trained lda model.now i want to find the new unseen document similarity with the corpus.
...
0
votes
1answer
15 views
Dissimilarity Matrix of non-metric proximity data
we currently have a coding exercise, where we are asked to implement Constant Shift Embedding (Paper). This in itself is not a big problem. For the algorithm, all you need is a symmetric non-zero ...
0
votes
1answer
22 views
How can I get an objective measure of song similarity?
I was browsing through ML project ideas and found an interesting one (just the problem statement ) which was: detecting if two songs are similar using lyrics. I found it to be an interesting idea but ...
2
votes
2answers
41 views
Random-Forest-based Similarity Matrix for clustering: how does it behave?
I am in the following context:
Data: static, baseline health data at the patient level, 40 features, sparse (~ 25 binary features with many 0 or many 1 + other categorical features)
Objective : to ...
2
votes
3answers
101 views
How to measure the similarity between two text documents?
Assume, I have 100 text documents, and I want to cluster those documents.
The first step is the construct pairwise similarity matrix 100X100 for the documents
My ...
0
votes
1answer
37 views
scipy.spatial.distance.mahalanobis return null values for some vectors in python
I am using scipy.spatial.distance.mahalanobis to calculate distance between two vectors but i'm getting null values for some vector I don't know why? null value is possible?
2
votes
1answer
68 views
How to measure the similarity between two images?
I have two group images for cat and dog. And each group contain 2000 images for cat and dog respectively.
My goal is try to cluster the images by using k-means.
Assume image1 is ...
1
vote
1answer
69 views
How to measaure the similarity between two series?
I'm confused about how to measure the similarity between two time series with the same length. For example, both time series are 2 hours in length and every 5 minutes a point. I really want to know ...
1
vote
0answers
100 views
Doc2vec most similar document to a query string
I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title ...
0
votes
0answers
22 views
Given an input phrase, is there a way I can find the most similar phrase within a document?
I am completing a task where I need to retrieve the corresponding values to a set of given labels from many legal contracts.
For example, one of the labels is "Floating rate payment dates" and it's ...
2
votes
1answer
29 views
Cluster Analysis - Comparing Same Individuals Clustered Across Different Datasets with different features
I have an interesting problem, and I think my Google is failing me since I can't find the same problem anywhere.
I have a set of individuals. I have 4 different datasets, with (some) to (all) of ...
2
votes
3answers
47 views
Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?
I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances.
Normally pairwise clustering is done like this: (...
1
vote
0answers
29 views
Clustering and graphing similarities of sentence subjects
I have a bunch of sentences. Each sentence is given a weight of how "close" it is to a particular subject.
Ex. "I love reading math books" Subjects for the above sentence = ...
3
votes
1answer
34 views
Calculating similarity where order matters
How can I calculate a similarity (coefficient) where the order of the items matters and something like the Jaccard index would not be useful.
Specifically, I'm interested in comparing ingredients. ...
1
vote
0answers
71 views
python - Identify variable in similar sentences
I'm looking to solve the following problem: I have a list of similar sentences as my dataset, and I want to be able to type a new sentence, which is also similar to the sentences in my dataset and ...
0
votes
0answers
33 views
Learning similarity of representations
I am interested in a framework for learning the similarity of different input representations based on some common context. I have looked into word2vec, SVD and other recommender systems, which does ...
2
votes
3answers
41 views
Match a two items from two different receipts
I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement).
Suppose I have ordered(PO) Wine:
White Wine
Red Wine
Rose Wine
And I receive ...
2
votes
2answers
83 views
How can I find similarities in two graphs?
Let's say that I have two 1 dimensional arrays, and when I plot the two arrays they look like this:
If you look at the top and bottom graphs, then you can see that the highlighted parts are very ...
2
votes
0answers
21 views
Measure of variety within list/cluster
I have a dataset of about 53000 points. It has been clustered twice, based on two sets of unrelated attributes. For the first clustering (clustering 1) I used DBScan, and it ended up with about 700 ...
0
votes
0answers
26 views
Quantify similarity between 2 multivariate time series samples
I am looking for methods to analyze the similarity between 2 multivariate samples of a time series data. I have tried and analyzed KS test, euclidean distance and correlation based methods. What are ...
-1
votes
2answers
37 views
Similarity measure before and after dimensionality reduction or clustering
I have a dataset with 500 000 samples, each sample contains 30 features.
The values of the features are in the range 0.0 to 1.0.
...
1
vote
1answer
36 views
How to find correlation between time-series of different units?
I have 3 time-series data.
NDVI(normalized difference vegetation index) mean
Precipitation
Temperature
All of these have their own unit.
Now I want to find similarity/correlation between NDVI and ...
1
vote
0answers
40 views
Sliding cross-correlation [closed]
I want to understand Timeseries shape similarity algorithm ( Shape-based distance aka SBD). I can't understand the statistics behind it and why it is better than DTW or other similarity measure. I'm ...
2
votes
1answer
17 views
When I would use a specific similarity coefficient over another?
Like using Jaccards over Dice.
I want real examples, of when I would prefer to use Jaccards, Dice, Cosine or any other similarity coefficient.
1
vote
1answer
143 views
Find all potential similar documents out of a list of documents using clustering [closed]
I'm working with the quora question pairs csv file which I loaded into a pd dataframe and isolated the qid and question so my questions are in this form :
...
0
votes
0answers
85 views
Normalizing Jaccard similarity scores in relation to differences in document length
The Jaccard similarity of two documents A and B can be defined as the size of their intersection (how many tokens are in both docs) divided by the size of their union (total number of tokens found in ...
2
votes
1answer
41 views
Similar students using Machine Learning
I have a student performance data, where I have marks of various subjects for the students and I want to find similar students with good marks in a particular subject using machine learning.
How do I ...
4
votes
2answers
216 views
how to apply similarity algorithm(or comparision) of over one million vectors with other one million vectors?
How can I apply similarity algorithm (or comparison) of over one million vectors with another one million vectors?
I am following this pyimage search tutorial but don't know how to scale up the ...
0
votes
1answer
87 views
Assigning a new document to a cluster based on keywords extracted and tf-idf
I have about 40 clusters of documents defined by a combination of k-means clustering algorithm and hand curation. For example, some of the clusters given by k-means are too noisy so they have been ...
1
vote
0answers
32 views
How to check the similarity between two transition matrixes
I have two transition matrixes in which the probabilities for the transition between each from state to each to states are ...
1
vote
0answers
61 views
How can I select a similarity threshold value for strings?
I have several large data sets of names (i.e., individuals, groups, and companies) where there is a lot of similar, but not quite the same data. For example, a corporation might be listed as "Acme ...
0
votes
2answers
135 views
Nearest Neighbors on mixed data types in high dimensions
I would like to be able to use nearest neighbors to attempt to find the most similar samples to a subclass of samples (think treated vs untreated) in a dataset with continuous, categorical, and text ...
0
votes
2answers
111 views
Use text similarity (cosine) instead of machine learning to classify companies into industries
I am building an industry classifier. I.e. classifying companies into industries based on a company's description. Each company can only have one industry.
I took 2000 companies and assigned them ...
1
vote
3answers
64 views
How to find similarity of two series over time containing periodic trends?
Considering the data is received from a streaming source each second.How to distinguish if both the line graphs 'look' same/different in real time, statically, like the picture given below
Edit:
1....
1
vote
0answers
17 views
Best way to find dissimilarity in a 6x2 DataFrame?
I'm new to data science and am currently learning different techniques that I can do with Python. Currently, I'm trying it out with Spotify's API for my own playlists.
The goal is to find the most ...
0
votes
0answers
16 views
Efficient Similarity Indexing and Searching in High Dimensions
I found this recent document, containing simple algorithm that properly indexes high dimensional vectors for binary search later.
Algorithm basically uses random partition forest that randomly ...
1
vote
1answer
23 views
Identifying documents similar to specific clusters
Through performing clustering on a set of 1,000,000 text documents, I have identified 100 clusters. I am particularly interested in, say, 10 of the clusters. Imagine, I now have an additional set of ...
1
vote
1answer
38 views
Recommender system that connect users with each other , should I go for content based or collaborative filtering?
I am trying to build a system where user come on the platform and he chooses a topic(predefined few topics) and then we connect him with any random online user who chooses the same topic. Then they ...
