Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
Join us in building a kind, collaborative learning community via our updated Code of Conduct.

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common ...

1
vote
1answer
6 views

Predicting Composition of Chemical Compounds

I have a dataset which has names of compounds and their compositions. Like below Sulphuric Acid=>[H,S,O] (Hydrogen, sulphur, oxygen) Oxalic Acid=>[H,C,O] Sodium Oxalate=>[Na,C,O] Potassium Sulphate=>[...
0
votes
1answer
11 views

Topic Segmentation - should it be done in Raw, TfIdf or Semantic Space?

Let's assume we have a collection of documents and wish to perform some unsupervised topic segmentation. As always, we will perform some preprocessing (including tokenization, accent-removal, ...
0
votes
0answers
5 views

Hierarchical Single-Link Clustering BGP Updates

I'm currently working in a project where I'm analyzing a series of BGP Updates obtained through RouteViews, I organized these updates in a dataframe and it comes with the next format: BGP4MP|...
-1
votes
0answers
11 views

Metric and Binary Variable in Cluster Analysis

we´re working on a seminar paper and have to conduct a cluster analysis (n = 130,000) with knn and k-means clustering. Our dataset consists mostly of binary variables such as gender. However, we have ...
1
vote
2answers
27 views

Evaluating clusters (e.g. built by kmean) using Random Forest

I have made clusters for my data set (1.5 million samples and 800 features) using k-mean. I am aware of internal indices for evaluating clusters. However, I was thinking about training a supervised ...
1
vote
0answers
16 views

Autoencoder ambivalent about order of input data?

The problem I'm working to solve is this: Given a musician's prerecorded free-form playing. I want to analyze each of the individual notes to determine how "in-rhythm" it is. See the graph in the ...
2
votes
1answer
80 views

Semi Supervised Learning without label propagation

I am trying to cluster some words by affinity. Using Word2Vec I obtained vector representation of every word that I can cluster with a normal unsupervised method. Of these words, though, I know the ...
0
votes
2answers
32 views

Is deduction, genetic programming, PCA, or clustering machine learning according to Tom Mitchells definition?

Tom M. Mitchell defines machine learning as A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at tasks in T, ...
3
votes
2answers
24 views

How can I compare classes from clusterings performed on two different data sets?

I have two data sets defined by real valued vectors, and I have performed clustering on both of them. Now I want to compare the classes to see how they map to each other. If I put the data sets ...
0
votes
2answers
20 views

Spectral vs Kmeans

What makes Spectral clustering better than Kmeans clustering? I understand that Kmeans clustering is the final step of Spectral. But why is it that the previous steps involved in Spectral clustering ...
0
votes
1answer
17 views

Grouping/clustering similar words python

I have a question regarding grouping of similar words for example I have list of words give below: artificialintelligence Artificial Intelligence AI Machine Learning ML Data Analytics Data & ...
1
vote
3answers
61 views

What is the best way to visualize the relationship two categorical variables

I am currently working on an ambulance dataset and one of my tasks is to find when a patient was misdiagnosed by the call dispatcher. I have two codes; a dispatch code(what the dispatcher believes is ...
0
votes
1answer
23 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
2
votes
2answers
18 views

Ordering scrambled 1D data sets by continuity

This is a cute little clustering problem that was probably solved a million times over, but I couldn't find a good reference for it. I have 20 1D datasets with 400 entries each. In the picture they ...
2
votes
1answer
46 views

Improve results of a clustering

I'm a beginner and I'm trying to do a clustering of a multi-sentence text, but my results are horrible. Any tips for me to improve my result? ...
0
votes
0answers
17 views

Samples that share same features but have different labels/output values

I have built a clustering model based on numerical data, specifically time series clustering. Let's say using sales quantities (over time) of different products. In other words identify different ...
-1
votes
0answers
10 views

Generalized end-to-end (GE2E) loss function

I will be using the GE2E loss function in my LSTM to create audio embeddings of audio samples. I am having difficulty with extracting the algorithm for GE2E loss function from this paper. Can I have ...
-1
votes
2answers
33 views

How to identify clusters after multiple runs?

Suppose I run an unsupervised clustering algorithm. After multiple runs, I find clusters and would like to know if the same cluster was found more than once. For example: I can figure out A-orange, ...
1
vote
2answers
18 views

Identify objects (bus) on the map based on coordinates (lat, lon) [closed]

Let's say I have an android app that frequently sends current GPS location of the user. If person is driving with bus, I can easily get GPS location of the bus and display it on the map and update it ...
0
votes
0answers
16 views

Classifying variable types on a list of variables

I have a list of around 700 variables which I need to perform a variable cleanup on. What complicates things is there are different numeric codes which flag an invalid value and these differ by the ...
0
votes
1answer
13 views

Speaker clustering/diarization

I am working on the problem of speaker clustering. I am using kmeans clustering. The ground truth cluster values and kmean cluster values do not correspond due to different methods of labelling(...
0
votes
0answers
19 views

Number of clusters with eigengap method in spectral clustering

I would like to find the number of clusters for spectral clustering which I could apply to my data. I've tried eigengap technique on a cosine similarity matrix and got the following plot: The gap ...
0
votes
1answer
20 views

Can cosine similarity be applied to multidimensional matrices?

I'm trying to find the similarity between two 4D matrices. Because cosine similarity takes the dot product of the input matrices, the result is inevitably a matrix. Is there a way to get a scalar ...
1
vote
1answer
25 views

Clustering 1-gram Strings

I have a big list of mail addresses (around one million) and I want to use/adapt/create an algorithm to find similarities between them (basically to cluster them). All the algorithms I've checked so ...
0
votes
1answer
16 views

Reconstructing original data points from t-SNE output

I have been trying to understand t-SNE for a while now and I have this very basic question on the comparison of PCA and t-SNE, on which I would really appreciate some help. In case of PCA suppose the ...
-1
votes
1answer
32 views

What is the most straightforward way to discover clusters in data? [closed]

I'm planning on extracting a number of word vector distances from a data set, and I want to be able to detect clusters within that set, with an undefined number of clusters that are dynamically ...
0
votes
2answers
47 views

Street address clustering?

I have a huge dataset of addresses. I have another data stream that contains addresses that I need to match against those in the original dataset. As all the addresses are user-provided, matching them ...
1
vote
2answers
53 views

clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
3
votes
2answers
42 views

Kmeans using silhouette_score

I am using silhouette_score to find the optimal k value. So I am running a for loop with a range of possible k values. I have added my code below. this program takes a very long time to run. Could you ...
0
votes
0answers
6 views

What defines a successful clustering in prototype clustering?

What defines a successful clustering in prototype clustering? Since upon studying and trying out prototype clustering, it seems that it's not that "robust" method in the sense that doing some ...
4
votes
3answers
61 views

Distance between very large discrete probability distributions

I have 192 countries where each country has some value for 1 million attributes which sum up to 1 (a discrete probability distribution). For any one country most of the values for the attributes are 0....
0
votes
0answers
9 views

Grading accounts behaviour on the 'scale of normality' using stats/ML methods

I am working on a project where I have data consisting of bank transactions performed by thousands of distinct users. Each user may have more than one bank account in the system. Each transaction has ...
-1
votes
1answer
19 views

How to solve online clustering problem

Suppose we have a clustering problem where data sample is of multi-dimension with a mix of numeric and categorical type. If the ...
0
votes
1answer
27 views

How to encode data with a feature having multidimensional features (colors)?

My dataset has around 20 features, one of which is colors(in string format). There are around 50 different colors. I have converted them to RGB, but now I want to encode the data in such a way that ...
2
votes
0answers
26 views

Clustering events in a sequence.

I've a sequence of recurring events I want to group together into representing different operation activities of the underlying process. 1) These events might potentially have an order in their ...
0
votes
2answers
36 views

How can we interpret biplot?

This is not a question as such but more likely to be verification (enhancement) of my current understanding. With the thought that it may help future visitors as well, I am taking liberty to make this ...
0
votes
1answer
22 views

Clustering 3D multivariate data

I am looking for a clustering procedure that will group a number of 3D points on the basis of their spatial relations and multivariate dimensions. Dimensions are mostly represented with (interval) ...
1
vote
1answer
24 views

Using PCA to cluster multidimensional data (RFM variables)

So i am performing k-means clustering on RFM variables (Recency, Frequency, Monetary). The RFM variables are in the form of quantiles (1-4). I used PCA and found the PCA components. I then used the ...
4
votes
2answers
102 views

What is Spectral clustering?

What is spectral clustering? I have little background in statistics. I have tried to search for notes online but they assume quite a lot of knowledge. Would be good if you are able to find some ...
0
votes
1answer
25 views

Customer-Product Analytics [closed]

I am new to Data Science and I want to make Customer Product Analytics for my company(bank). I can have a data of customers, their income, daily transactions, average balance etc and what product(...
0
votes
2answers
20 views

EM clustering with missing and misspelling data

I am currently working on a project that requires me to cluster the unlabeled input. The records contain personal information such as name, DOB, height, sex, etc. We need to cluster the same person in ...
0
votes
1answer
25 views

How to find unknown number of clusters in circular data?

I have some 1 dimensional data. Each record in the data is a specific time of the day. In order to cluster it I projected the data onto a circle of radius 1 unit. Now I need to find clusters in this ...
1
vote
0answers
20 views

What is the difference between K-Means & Self Organized Maps?

It seems they both perform clustering. They both reduce the dimensionality of the input data and classify further inputs based upon their distance/similarity to the center points. These points then ...
0
votes
2answers
64 views

What approach other than Tf-Idf could I use for text-clustering using K-Means?

I am working on a text-clustering problem. My goal is to create clusters with similar context, similar talk. I have around 40 million posts from social media. To start with I have written clustering ...
1
vote
1answer
17 views

Clustering evaluation metrics with subquadratic time complexity

It exists many evaluation metrics but often they are quadratic or more on number of data points preventing any application on massive data sets as RAND or Silhouette indexes. For the moment i used : ...
3
votes
2answers
52 views

Extracting useful features for k-means clustering

So say suppose I have a data-set with features being either present or not i.e. 0or 1. Now I want to identify the features which ...
1
vote
2answers
52 views

Similarity between two scatter plots [closed]

I would like to know if there is a metric used to compute the similarity between two scatter plots?
0
votes
2answers
74 views

How to cluster histograms or density distributions?

I have exactely no idea of where to start when it come to cluster distribution and find out similar the similar one. is there a package in R that do the job ?
0
votes
1answer
32 views

Clustering with groups in data related to cluster label

I want to predict which device got used in which room. Therefore I've got device and sensor data. My idea was to create a feature vector lie this: ...