Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
157
votes
13answers
200k views

K-Means clustering for mixed numeric and categorical data

My data set contains a number of numeric attributes and one categorical. Say, NumericAttr1, NumericAttr2, ..., NumericAttrN, CategoricalAttr, where ...
27
votes
4answers
29k views

Is it necessary to standardize your data before clustering?

Is it necessary to standardize your data before cluster? In the example from scikit learn about DBSCAN, here they do this in the line: ...
2
votes
1answer
1k views

Cluster documents and identify the prominent document in the cluster?

I have a set of documents as given in the example below. ...
5
votes
1answer
1k views

Clusering based on categorical variables?

I am working on a project and currently experimenting cluster analysis. The dataset is mainly categorical variables and discrete numbers. Please pardon my poor programming skills as I am not very ...
22
votes
2answers
8k views

How to deal with time series which change in seasonality or other patterns?

Background I'm working on a time series data set of energy meter readings. The length of the series varies by meter - for some I have several years, others only a few months, etc. Many display ...
2
votes
2answers
7k views

Clustering users based on buying behaviour

I have a set of data which indicates purchase transaction of users (~1 million records). User can have more than 1 purchase across time. Data is spread over 6-7 months. Attributes that I have are ...
4
votes
3answers
11k views

How to classify and cluster this time series data

I have post already the question few months ago about my project that I'm starting to work on. This post can be see here: Human activity recognition using smartphone data set problem Now, I know ...
6
votes
3answers
27k views

How to test accuracy of an unsupervised clustering model output?

I am trying to test how well my unsupervised K-Means clustering properly clusters my data. I have an unsupervised K-Means clustering model output (as shown in the first photo below) and then I ...
0
votes
1answer
115 views

How to evaluate clusters base on a label?

I have a data set that has an attribute(A) with 300 different nominal values. Attribute A has a lot of noise. I decide to cluster my data based on other attributes that related to A. I hope to reach ...
59
votes
8answers
71k views

Clustering geo location coordinates (lat,long pairs)

What is the right approach and clustering algorithm for geolocation clustering? I'm using the following code to cluster geolocation coordinates: ...
23
votes
1answer
23k views

Word2Vec vs. Sentence2Vec vs. Doc2Vec

I recently came across the terms Word2Vec, Sentence2Vec and Doc2Vec and kind of confused as I am new to vector semantics. Can someone please elaborate the differences in these methods in simple words. ...
10
votes
4answers
21k views

Clustering for mixed numeric and nominal discrete data

My data includes survey responses that are binary (numeric) and nominal / categorical. All responses are discrete and at individual level. Data is of shape (n=7219, p=105). Couple things: I am ...
9
votes
5answers
22k views

Clustering with cosine similarity

I have a large data set and a cosine similarity between them. I would like to cluster them using cosine similarity that puts similar objects together without needing to specify beforehand the number ...
15
votes
2answers
7k views

K-means vs. online K-means

K-means is a well known algorithm for clustering, but there is also an online variation of such algorithm (online K-means). What are the pros and cons of these approaches, and when should each be ...
6
votes
2answers
6k views

For which real world data sets does DBSCAN surpass K-means.?

For clustering, DBSCAN surpass k-means in terms of handling arbitrary shape data sets. In the most published papers about density based clustering, the experiments are performed with synthetic data ...
8
votes
3answers
1k views

Algorithm for segmentation of sequence data

I have a large sequence of vectors of length N. I need some unsupervised learning algorithm to divide these vectors into M segments. For example: K-means is not suitable, because it puts similar ...
19
votes
4answers
7k views

Clustering based on similarity scores

Assume that we have a set of elements E and a similarity (not distance) function sim(ei, ej) between two elements ei,ej ∈ E. How could we (efficiently) cluster the elements of E, using sim? k-means,...
8
votes
2answers
2k views

Fitting lines through large point clouds

I have a large set of points (order of 10k points) formed by particle tracks (movement in the xy plane in time filmed by a camera, so 3d - 256x256px and ca 3k frames in my example set) and noise. ...
9
votes
2answers
4k views

Human activity recognition using smartphone data set problem

I'm new to this community and hopefully my question will well fit in here. As part of my undergraduate data analytics course I have choose to do the project on human activity recognition using ...
6
votes
1answer
2k views

Similarity measure for multivariate time series with heterogeous length and content

I am interested in clustering multivariate N time series of T'values' each(different lengths) using python. Each variable have many trends and values which are simultaneously numeric and nominal. A ...
3
votes
2answers
1k views

Cluster documents based on topic similarity

I have set of documents where I have assigned topics per each document. E.g., Topics of document 1 -> 1.0 Science, 1.0 politics, 0.8 History, 0. 8 Information and Technology Now I want to cluster ...
1
vote
1answer
93 views

How to validate a clustering model without a ground truth?

Im dealing with a dataset (text messages about source code comments) that are not labeled. I don't have a assumption about the implicits classes in this dataset. I want to discovery (by clustering) ...
6
votes
3answers
510 views

How to give a higher importance to certain features in a (k-means) clustering model?

I am clustering data with numeric and categorical variables. To process the categorical variables for the cluster model, I create dummy variables. However, I feel like this results in a higher ...
5
votes
2answers
17k views

Perform k-means clustering over multiple columns

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not ...
4
votes
5answers
4k views

Best approach for this unsupervised clustering problem with categorical data?

I'm a software engineer new to Machine Learning. I've read about basic non-supervised techniques like k-means and hierarchical clustering and now I'm trying to put them into practice with a basic ...
4
votes
4answers
8k views

Can I use unsupervised learning followed by supervised learning?

I have a question about classifying documents using supervised learning and unsupervised learning. For example: - I have a bunch of documents talking about football. As we know, football has a ...
1
vote
1answer
107 views

Devices behavior in one continuous variable vs events rate

I have devices on which I have time series data of one continuous variable. I have to evaluate the relation between the profile of that variable on those devices and "events". Those events are given ...
1
vote
1answer
97 views

Clustering 1-gram Strings

I have a big list of mail addresses (around one million) and I want to use/adapt/create an algorithm to find similarities between them (basically to cluster them). All the algorithms I've checked so ...
0
votes
2answers
756 views

Hierarchical Clustering customized Linkage function

In my clustering project, I need to customize the linkage function, so that after each cluster merging I can update the inter-cluster distance in my own way. Currently I'm using scikit-learn ...
0
votes
3answers
2k views

How to cluster histograms or density distributions?

I have no idea where to start when it come to cluster distribution and finding out similar the similar one. Is there a package in R that does the job?
6
votes
2answers
869 views

Clustering Observations by String Sequences (Python/Pandas df)

I have a dataset consisting of approximately 2 million unique observations. It was initially a set of ID's and URLs. The goal is to cluster the ID's based on the URLs looked at. I transformed both ...
5
votes
1answer
72 views

How do I interpret my result of clustering?

I am working on a clustering problem. I have 11 features. My complete data frame has 70-80% zeros. The data had outliers that I capped at 0.5 and 0.95 percentile. However, I tried k-means (python) on ...
4
votes
1answer
657 views

What's the difference between finding the average Euclidean distance and using inertia_ in KMeans in sklearn?

I've found two different approaches online when using the Elbow Method to determine the optimal number of clusters for K-Means. One approach is to use the following code: ...
4
votes
1answer
969 views

Clustering pair-wise distance dataset

I have generated a dataset of pairwise distances as follows: id_1 id_2 dist_12 id_2 id_3 dist_23 I want to cluster this data so as to identify the pattern. I ...
3
votes
3answers
2k views

Clustering algorithm for a distance matrix

I have a similarity matrix between N objects. For each N objects, I have a measure of how similar they are between each others - 0 being identical (the main diagonal) and increasing values as they get ...
3
votes
1answer
92 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
3
votes
2answers
3k views

How can we evaluate DBSCAN parameters?

yes, DBSCAN parameters, and in particular the parameter eps (size of the epsilon neighborhood). In the documentation we have a "Look for the knee in the plot". Fine, but it requires a visual analysis....
3
votes
1answer
124 views

Transition from clustering to classification?

To date, I have done several ad-hoc text clustering projects which use combinations of topic modeling, k-means, and other algorithms. Basically, the point of these projects was to produce themes for ...
2
votes
2answers
4k views

Overfitting in an unsupervised technique

I am trying to understand if over-fitting can happen in an unsupervised technique like kmeans clustering. Could someone help me understand if and how this would happen? Thanks.
2
votes
3answers
146 views

Memory-efficient metric calculation for ultra high dimensional data

I am preparing for clustering the data which can be only represent as a extremely sparse binary vectors. Each of the objects is represented by a large set the binary features ($10^3$ ~ $10^6$), each ...
2
votes
4answers
73 views

Unsupervised clustering without of Data which is supposed to be on a linear function

When I have a dataset where each datum has x and y, and the (x,y) has a relation of one of <...
1
vote
2answers
684 views

how to handle outliers for clustering algorithms?

I am wondering what's the best way to handle outliers when using non-supervised clustering algorithms? Thanks!
1
vote
2answers
484 views

Intra-cluster similarity metric

I have some observations belonging to groups and I would like to compute the similarity of them within different groups in order to tell which observations, within specific groups, have similar ...
1
vote
1answer
189 views

Text classification and clustering with complete date imbalance

I have a set of scientific papers of authors who have common research interests from PUBMED and I would like to: Clustering papers and extracting features from them in order to find other authors ...
1
vote
1answer
103 views

How to evaluate clusters base on an attribute of the dataset? [duplicate]

I have a data set of persons with attribute job that have 300 different nominal value. attribute job have a lot of noise. I decide to cluster my data base on other attribute (other feature of person) ...