Questions tagged [similarity]
The similarity tag has no usage guidance.
198
questions
0
votes
1answer
32 views
how to build word2vec content based recommendation?
I am building a content-based recommendation system for hotel accommodation. I have a hotel name, hotel description and location. I combined hotel name, description and location. Then, applied NLP and ...
0
votes
0answers
14 views
Is there a way to normalize a similarity matrix by row and column in a way such that only one entry per row or column is approximately 1
I am computing similarities between 2 vectors. My goal is to have approximately 1 matching sample with similarity ~1, for each sample, without having any samples that are similar to many other samples....
3
votes
1answer
123 views
Comparing one small dataset with a big dataset for similar records
I create a varying small dataset (dataset: X) with 500 records in each query. Everytime I need to compare the dataset with a bigger one (dataset: A) (15 milion records) to find similar (or semi-...
0
votes
0answers
42 views
Semantic Search Help
There is a problem we are trying to solve where we want to do semantic search on our set of data,
i.e we have a domain specific data (example: sentences talking about automobiles)
Our data is just a ...
1
vote
0answers
20 views
Options to find the most similar question in a dataset of question-answer pairs?
I am building a chatbot that will only handle FAQs, but these FAQs are very specific to an organisation, so I cannot use any existing off-the-shelf solutions, or connect to question-answering APIs.
I ...
1
vote
1answer
31 views
Calculate Similarity using User's Personal Data?
I want to find out which users are similar to each other using their personal/organisational data, such as department, company, site, etc.
I have this data in a boolean format, as shown below:
...
1
vote
0answers
15 views
document similarity when the document size less than 30 tokens?
I was solving a problem to compare 3 million, 2018 documents against the 2019 documents. There are three text attributes to be compared from one item against the other. I used Latent Semantic Indexing ...
2
votes
1answer
40 views
Approach to semantic similarity between documents
I was wondering what approach people would take, or point me in the right direction on this challenge I have set myself. I am pretty new at this, I have covered some area but want to expand my ...
1
vote
2answers
78 views
How can we perform STS(Semantic Textual Similarity) on UnSupervised dataset using Deep Learning?
How to implement STS(Semantic Textual Similarity) on unlabelled dataset. Dataset column contains Unique_id, text1(contains paragraph), text2(contains paragraph).
Ex: Column representation: Unique_id ...
4
votes
2answers
51 views
Cluster elements that appear in the same lists
Suppose I have a multitude of sets with (unordered) combinations of elements and I want to determine which elements tend to appear together.
For example
Given the following sets:
...
0
votes
1answer
23 views
looking for approaches to detecting outliers in individuals unequal sequential time series
I am looking for approaches related to outlier detection in time series.
Example:
A person visits hospital overtime on multiple bases and there are some measurements made (bmi, blood_pressure, ...
0
votes
0answers
13 views
How to get a percentage of similarities of new data added based on model trained using a huge training set in machine learning
I need to find similarities in newly added data using machine learning.
We have a huge data set with more than 200 000 rows having following information:
Name
D.O.B
City (From predefined list)
Phone ...
0
votes
3answers
375 views
Cosine similarity vs The Levenshtein distance
Cosine similarity vs The Levenshtein distance
I wanted to know what is the difference between them and in what situations they work best?
As per my understanding:
Cosine similarity is a measure of ...
2
votes
2answers
290 views
Similarity of words using BERTMODEL
I want to find the similarity of words using the BERT model within the NER task. I have my own dataset so, I don't want to use the pre-trained model.
I do the following:
...
0
votes
2answers
99 views
Looking for similar items in a large data set
I have a large database of people and I want to show a small number of people who are similar to each person in the database. So if one of the people was Wolfgang Mozart I would want to show Beethoven,...
2
votes
1answer
89 views
How to build a symmetric similarity model on top of embeddings?
I have two equal length vectors that come out of two identical embedding layers.
I want to calculate their similarity, and I don't trust the embedding layer enough to just use dot product (e.g. it's ...
0
votes
0answers
8 views
How to draw a support set when classifying using Siamese networks without performing one shot learning?
How to perform classification on a test set with Siamese networks when I cannot afford to draw the support set from the test set itself?
Possible options which come to my mind are:
KNN using samples ...
0
votes
2answers
262 views
Question about Similarity vs Dissimilarity Matrix
Right now, I'm working on a coming up with a similarity vs dissimilarity matrix for a set of data points for a clustering algorithm. My question is, if I want to use one of the many clustering ...
0
votes
0answers
5 views
How to identify similar datasets based on multiple temporal variables?
I have a dataset that describes movie releases, i.e. day by day number of seats allocated to the movie and number of seats sold at different geographic locations. This dataset looks something like ...
0
votes
1answer
46 views
Clustering stores based on weekly data
I have 1 year transaction level data aggregated at a weekly level for 1000 different stores. I want to cluster similar stores based on 8 variables such as sales, customer count etc. The concern is ...
1
vote
1answer
32 views
How to find similar points to a positive set when you don't have any negative set?
The task I'm used to do is the following. A client comes to see me with a set of clients (called positive companies) and he wants me to find other similar prospects. Usually, he also gives me a set of ...
5
votes
2answers
1k views
Text similarity with sentence embeddings
I'm trying to calculate similarity between texts with various lengths. My current approach is following:
Using Universal Sentence Encoder, I convert text to a set of vectors.
I average these vectors ...
0
votes
0answers
13 views
Matching resumes with jobs in a NoSQL database
I have a series of user resumes and would like to find the best matching jobs per resume, the task seems simple at first except I have a nosql db holding the jobs (1m+), my initial idea was to add a ...
-1
votes
1answer
171 views
Word Embeddings with TFIDF vectorizer
I am a beginner in machine learning.
I have a large corpus of texts, divided into thematic groups. I would like to get idf values for the whole corpus, and then apply it on each group before ...
0
votes
0answers
55 views
Modifying BERT sentence encodings
I'm using BERT to encode sentences.
The sentences I'm encoding are quite similar, meaning they all belong to the same overall topic. Therefor, I am using another parameter for measuring similarity.
...
1
vote
2answers
48 views
Similarity Measure of Simulated Time Series vs Observed time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
1
vote
0answers
13 views
Looking for an algorithm that compute similarity between a phrase and possible combination of tokens
I want to find similarity between a phrase and possible combination of tokens that may form the phrase.
For example, phrase = 'sea surface water'
Possible token = ['sea','surface land', 'surface ...
0
votes
0answers
12 views
What options do I have for measuring similarities by using vectors generated from texts?
I have a data set which contains vectors generated from subtitles, I want to measure the similarity between each pair of the observation. Now I have tried L1, L2, cosine similarity and Mahalanobis ...
0
votes
0answers
28 views
Is it necessary to perform Z-score or Min-Max normalization on L1-normalized data?
I have a dataset which contains vectors that generated from subtitles and have been L1 normalised, I want to calculate cosine similarity & Euclidean distance, I thought it is better if I use Z-...
2
votes
0answers
115 views
Similarity Measure Time Series
In my work I have an observed Time Series and Simulated ones. I want to compare the Light Curves and check for similarityto find out which simulated curve fits best respectivley which parameters ...
0
votes
0answers
15 views
supervised similarity scoring system - recommender system
I have a dataset of movie collections with 10-15 features describing each movie. I also have a dataset of user ratings of the similarity between some random pairs of movies.
Using both of these data ...
0
votes
0answers
25 views
What is the meaning of constraint two neural networks against each other?
I have two identical autoencoders with the same number of layers and parameters. I would like to find the similarity between two images from different domains such as an image captured from the camera ...
6
votes
2answers
97 views
How to compare different similarity measurements in text clustering?
I have a dataset which contains vectors generated from subtitles (each column represents a genre, each row is a movie name), my purpose is to find the most similar movie titles, I want to use ...
0
votes
0answers
5 views
How to validate model which outputs movies that are similar with no training data
At a high level, my code takes a wide variety of movie related features and computes a large cosine similarity matrix and assesses which movies are most similar.
I don't have any validation data so ...
0
votes
0answers
92 views
Cluster based on both positions and similarity scores
I have a dataframe position giving me the x and y positions of 87 points.
I also have a 87 x 87 similarity matrix giving me the pairwise similarity scores between ...
1
vote
0answers
41 views
Measuring the similarity between a numeric data matrix and one or more categorical variables?
Given a numeric data matrix $A$ of size $n \times p$, which each row represents an observation along $p$ variables, and a second categorical data matrix $M$ of size $n \times z$, where each row ...
1
vote
1answer
354 views
How do I get similarity with autoencoders
I have build an autoencoder to extract from a very high dimensional (200 dimensions) space a smaller but significant representation (16 dimensions).
Now that I have these "encoded" vectors, I would ...
2
votes
2answers
108 views
How can I find correlation between features?
The problem
I want to figure out how routers correlate between each other. Like, if a specific error occurred in router A, and almost at the same time the error occurs in router B, they probably have ...
0
votes
0answers
40 views
Constrained Deep Learning
What are the ways to perform constrained optimization of weights in deep model? I am working on deep similarity learning, and many methods are based on constrained optimization; the vast majority of ...
0
votes
0answers
65 views
what will be the best algorithm for find similar users based on user features , betwenn knn, euclidian or jaccard?
I have user data table containing user data like university, gender, city, id etc. I also have a matrix data-set like this .
[
1
vote
1answer
41 views
Predicting similarity between nouns like university names and tech companies?
I am trying to extract entities like university studied at and tech companies from resumes , I have a list of popular universities and companies and I want to find out which university best matches ...
0
votes
0answers
19 views
Any there databases with native support for applying a NN model to produce search rankings?
The Situation: I have a simple neural net with an input vector that consists of the euclidean distance between the attributes of two cars. (For example the attributes would include wheel size, car ...
0
votes
0answers
450 views
Semantic Similarity in Universal Sentence Encoder
I am currently using Universal Sentence Encoder to embed certain sentences which I would then feed to a deep learning model to do some prediction, but just to test whether the universal sentence ...
0
votes
0answers
31 views
Features Vectors in embedding space
I have a bunch of users, each of them with about 100 features.
My goal is to create an embedded space to compute the "distance" between users.
Also, I want to be able to visualize it with Tensorboard (...
2
votes
2answers
304 views
Similarity score: Can Sklearn SVR predict values greater than 1 and less than 0?
I am using svm.SVR() from scikit-learn to apply Logistic Regression on my training data to solve a similarity problem. Using GridSearchCV, I am finding the best ...
4
votes
6answers
216 views
Distance between users
I want to compute the "distance" between users in order to return the top n similar users, for any given user.
For each user a have a bunch of features.
This is close to a recommendation system, ...
1
vote
1answer
51 views
identify similarities in a dataset
I have a dataset of customers:
age height weight eye_colour
30 174 74 Nan
20 191 71 Nan
28 165 56 Brown
...
I would like to ...
1
vote
1answer
35 views
how to match a sentence to a cluster of keywords?
I have a classification problem. I have clusters called 'Experience', 'Education', 'Abilities' . The labelled data (72,000+ entries with all clusters together) with two columns looks like below.
<...
1
vote
1answer
256 views
Jaccard similarity calculate similarity
It is not clear to me how to calculate similarity between two products from the example.
How do they calculate that?
1
vote
2answers
30 views
How can I measure if a population has the same distribution as other?
Is there a population similarity index of some kind which could help me determine if two populations in two different datasets are the same or at least similar?
The datasets have the exact same ...
