Data Science

Questions tagged [similarity]

Ask Question

The tag has no usage guidance.

175 questions
Newest
Active
Bountied
Unanswered
Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

What options do I have for measuring similarities by using vectors generated from texts?

I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
0answers
13 views

Is it necessary to perform Z-score or Min-Max normalization on L1-normalized data?

I have a dataset which contains vectors that generated from subtitles and have been L1 normalised, I want to calculate cosine similarity & Euclidean distance, I thought it is better if I use Z-...
1
vote
0answers
23 views

Similarity Measure Time Series

In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
0
votes
0answers
12 views

supervised similarity scoring system - recommender system

I have a dataset of movie collections with 10-15 features describing each movie. I also have a dataset of user ratings of the similarity between some random pairs of movies. Using both of these data ...
0
votes
0answers
21 views

What is the meaning of constraint two neural networks against each other?

I have two identical autoencoders with the same number of layers and parameters. I would like to find the similarity between two images from different domains such as an image captured from the camera ...
6
votes
2answers
49 views

How to compare different similarity measurements in text clustering?

I have a dataset which contains vectors generated from subtitles (each column represents a genre, each row is a movie name), my purpose is to find the most similar movie titles, I want to use ...
0
votes
0answers
5 views

How to validate model which outputs movies that are similar with no training data

At a high level, my code takes a wide variety of movie related features and computes a large cosine similarity matrix and assesses which movies are most similar. I don't have any validation data so ...
0
votes
0answers
21 views

Cluster based on both positions and similarity scores

I have a dataframe position giving me the x and y positions of 87 points. I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
1
vote
0answers
9 views

Measuring the similarity between a numeric data matrix and one or more categorical variables?

Given a numeric data matrix $A$ of size $n \times p$, which each row represents an observation along $p$ variables, and a second categorical data matrix $M$ of size $n \times z$, where each row ...
1
vote
1answer
42 views

How do I get similarity with autoencoders

I have build an autoencoder to extract from a very high dimensional (200 dimensions) space a smaller but significant representation (16 dimensions). Now that I have these "encoded" vectors, I would ...
2
votes
2answers
54 views

How can I find correlation between features?

The problem I want to figure out how routers correlate between each other. Like, if a specific error occurred in router A, and almost at the same time the error occurs in router B, they probably have ...
0
votes
0answers
30 views

Constrained Deep Learning

What are the ways to perform constrained optimization of weights in deep model? I am working on deep similarity learning, and many methods are based on constrained optimization; the vast majority of ...
0
votes
0answers
15 views

what will be the best algorithm for find similar users based on user features , betwenn knn, euclidian or jaccard?

I have user data table containing user data like university, gender, city, id etc. I also have a matrix data-set like this . [
1
vote
1answer
30 views

Predicting similarity between nouns like university names and tech companies?

I am trying to extract entities like university studied at and tech companies from resumes , I have a list of popular universities and companies and I want to find out which university best matches ...
0
votes
0answers
11 views

Any there databases with native support for applying a NN model to produce search rankings?

The Situation: I have a simple neural net with an input vector that consists of the euclidean distance between the attributes of two cars. (For example the attributes would include wheel size, car ...
0
votes
0answers
88 views

Semantic Similarity in Universal Sentence Encoder

I am currently using Universal Sentence Encoder to embed certain sentences which I would then feed to a deep learning model to do some prediction, but just to test whether the universal sentence ...
0
votes
0answers
18 views

Features Vectors in embedding space

I have a bunch of users, each of them with about 100 features. My goal is to create an embedded space to compute the "distance" between users. Also, I want to be able to visualize it with Tensorboard (...
2
votes
2answers
87 views

Similarity score: Can Sklearn SVR predict values greater than 1 and less than 0?

I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve a similarity problem. Using GridSearchCV, I am finding the best ...
4
votes
6answers
161 views

Distance between users

I want to compute the "distance" between users in order to return the top n similar users, for any given user. For each user a have a bunch of features. This is close to a recommendation system, ...
1
vote
1answer
22 views

identify similarities in a dataset

I have a dataset of customers: age height weight eye_colour 30 174 74 Nan 20 191 71 Nan 28 165 56 Brown ... I would like to ...
1
vote
1answer
24 views

how to match a sentence to a cluster of keywords?

I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below. <...
1
vote
1answer
58 views

Jaccard similarity calculate similarity

It is not clear to me how to calculate similarity between two products from the example. How do they calculate that?
1
vote
2answers
30 views

How can I measure if a population has the same distribution as other?

Is there a population similarity index of some kind which could help me determine if two populations in two different datasets are the same or at least similar? The datasets have the exact same ...
3
votes
4answers
46 views

Best way to combine two similar document

I have f.ex.: two news-articles that report the same event. However, these two text are similar BUT not the same. I would like to combine these two texts creating one text that contains only the most "...
2
votes
1answer
142 views

Jaccard Similarity with Binary Data

I have 5400 rows of data and 3211 columns of attributes. The first 4 columns are ID/Name/ParentID/ObjectType - the rest of the 3207 columns are the attributes that are to be used for similarity ...
1
vote
0answers
64 views

similarity measures for categorical data in data mining

I am working on my assignment in which i have to mention 5 similarity measures for categorical and continuous data in data mining.As a beginner i tried my best and found SQUARE DISTANCE,EUCLIDEAN AND ...
0
votes
0answers
11 views

how to get similary of unseen document with whole corpus

I have a trained lda model.now i want to find the new unseen document similarity with the corpus. ...
0
votes
1answer
15 views

Dissimilarity Matrix of non-metric proximity data

we currently have a coding exercise, where we are asked to implement Constant Shift Embedding (Paper). This in itself is not a big problem. For the algorithm, all you need is a symmetric non-zero ...
0
votes
1answer
22 views

How can I get an objective measure of song similarity?

I was browsing through ML project ideas and found an interesting one (just the problem statement ) which was: detecting if two songs are similar using lyrics. I found it to be an interesting idea but ...
2
votes
2answers
69 views

Random-Forest-based Similarity Matrix for clustering: how does it behave?

I am in the following context: Data: static, baseline health data at the patient level, 40 features, sparse (~ 25 binary features with many 0 or many 1 + other categorical features) Objective : to ...
2
votes
3answers
284 views

How to measure the similarity between two text documents?

Assume, I have 100 text documents, and I want to cluster those documents. The first step is the construct pairwise similarity matrix 100X100 for the documents My ...
0
votes
1answer
43 views

scipy.spatial.distance.mahalanobis return null values for some vectors in python

I am using scipy.spatial.distance.mahalanobis to calculate distance between two vectors but i'm getting null values for some vector I don't know why? null value is possible?
2
votes
1answer
127 views

How to measure the similarity between two images?

I have two group images for cat and dog. And each group contain 2000 images for cat and dog respectively. My goal is try to cluster the images by using k-means. Assume image1 is ...
1
vote
1answer
104 views

How to measaure the similarity between two series?

I'm confused about how to measure the similarity between two time series with the same length. For example, both time series are 2 hours in length and every 5 minutes a point. I really want to know ...
1
vote
0answers
126 views

Doc2vec most similar document to a query string

I'm working on a project and I created doc2vec representation of different academics which include their patents and publications etc. For each publication and patent I have information such as title ...
0
votes
0answers
23 views

Given an input phrase, is there a way I can find the most similar phrase within a document?

I am completing a task where I need to retrieve the corresponding values to a set of given labels from many legal contracts. For example, one of the labels is "Floating rate payment dates" and it's ...
3
votes
1answer
32 views

Cluster Analysis - Comparing Same Individuals Clustered Across Different Datasets with different features

I have an interesting problem, and I think my Google is failing me since I can't find the same problem anywhere. I have a set of individuals. I have 4 different datasets, with (some) to (all) of ...
2
votes
3answers
55 views

Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?

I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances. Normally pairwise clustering is done like this: (...
1
vote
0answers
29 views

Clustering and graphing similarities of sentence subjects

I have a bunch of sentences. Each sentence is given a weight of how "close" it is to a particular subject. Ex. "I love reading math books" Subjects for the above sentence = ...
4
votes
2answers
44 views

Calculating similarity where order matters

How can I calculate a similarity (coefficient) where the order of the items matters and something like the Jaccard index would not be useful. Specifically, I'm interested in comparing ingredients. ...
1
vote
0answers
74 views

python - Identify variable in similar sentences

I'm looking to solve the following problem: I have a list of similar sentences as my dataset, and I want to be able to type a new sentence, which is also similar to the sentences in my dataset and ...
0
votes
1answer
44 views

Learning similarity of representations

I am interested in a framework for learning the similarity of different input representations based on some common context. I have looked into word2vec, SVD and other recommender systems, which does ...
2
votes
3answers
45 views

Match a two items from two different receipts

I have two different invoices or receipts. One is a Purchase order one is something like a receipt(acknowledgement). Suppose I have ordered(PO) Wine: White Wine Red Wine Rose Wine And I receive ...
2
votes
2answers
135 views

How can I find similarities in two graphs?

Let's say that I have two 1 dimensional arrays, and when I plot the two arrays they look like this: If you look at the top and bottom graphs, then you can see that the highlighted parts are very ...
2
votes
0answers
22 views

Measure of variety within list/cluster

I have a dataset of about 53000 points. It has been clustered twice, based on two sets of unrelated attributes. For the first clustering (clustering 1) I used DBScan, and it ended up with about 700 ...
0
votes
0answers
30 views

Quantify similarity between 2 multivariate time series samples

I am looking for methods to analyze the similarity between 2 multivariate samples of a time series data. I have tried and analyzed KS test, euclidean distance and correlation based methods. What are ...
-1
votes
2answers
42 views

Similarity measure before and after dimensionality reduction or clustering

I have a dataset with 500 000 samples, each sample contains 30 features. The values of the features are in the range 0.0 to 1.0. ...
1
vote
1answer
46 views

How to find correlation between time-series of different units?

I have 3 time-series data. NDVI(normalized difference vegetation index) mean Precipitation Temperature All of these have their own unit. Now I want to find similarity/correlation between NDVI and ...
1
vote
0answers
42 views

Sliding cross-correlation [closed]

I want to understand Timeseries shape similarity algorithm ( Shape-based distance aka SBD). I can't understand the statistics behind it and why it is better than DTW or other similarity measure. I'm ...
2
votes
1answer
22 views

When I would use a specific similarity coefficient over another?

Like using Jaccards over Dice. I want real examples, of when I would prefer to use Jaccards, Dice, Cosine or any other similarity coefficient.

1 2 3 4