Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

1,309 questions
Filter by
Sorted by
Tagged with
10 views

Clustering with missing (incomplete) data by design

I'm a psychology PhD student and trying to formulate a research plan. As an example, let's say I want to know which subjects are "cost efficient" to study, by clustering subjects. I'd have a ...
11 views

How to fine tune n_components parameter in UMAP?

I am using UMAP for clustering. However I can't find any information about methods to fine tune n_components parameter (which is very important). As good as I understand I can't use explained variance ...
• 143
1 vote
13 views

How to understand to what maximum size you can reduce the dimension of data and avoid the curse of the dimensionality?

i have a question, maybe someone could help me. I use t-sne (also tried umap) to reduce the dimensionality of the text embeddings dataset (size of embedding 300). after that I will cluster using ...
• 143
8 views

How to get optimal number of clusters considering 1 and N clusters?

I want to create clusters using kmeans for subsets of a dataset and i created a grid search function to get the optimal amount using silhouette score, but it seems that silhouette from sklearn does ...
1 vote
24 views

Which clustering/partitioning algorithms can operate on arbitrary pairwise similarity or distance matrices?

I'm relatively new to cluster analysis, and I'm exploring options for general-purpose, non-hierarchical, strict partitioning of data based on a pre-computed $N\times N$ pairwise similarity matrix. ...
• 111
1 vote
7 views

Clustering 3D with survey data

I have data obtained from a survey and I would like to perform a 3D clustering of the individuals who have answered the survey based on 3 of the questions they have answered: Are you satisfied with ...
• 11
1 vote
14 views

How to check if customer segmentation/classification is correct?

I'm looking for the best way to check if our customers have the correct segmentation label. (New) customers are given a segment label at creation, mainly based on the information available at that ...
20 views

How to measure similarities between two datasets with same features?

I have multiple datasets with the same features, a few numerical and a few categorical. The only difference is that they are market behavior for different countries. I wanted to know if there is a way ...
64 views

Using k-means to create labels for supervised learning

I want to know if the following is a valid approach to create labels, if I have measurements under some conditions, and the conditions are similar but never exactly the same. This doesn't correspond ...
• 23
20 views

I need help with which features to use for clustering

I am using this dataset: https://www.kaggle.com/datasets/sobhanmoosavi/us-accidents and so far I have successfully cleaned the dataset as well as reduced the size of the features and records. I have ...
8 views

density vs shape

Can you explain this passage please: "A key feature of HDBScan is that it clusters data of varying density, this is in comparison to DBScan, which tend to cluster data of varying shapes only.&...
8 views

Multi-attribute pandas data frame to a weighted network - how to manage attribute values as weights?

I have a pandas data frame that includes n rows, indicating people, with m columns, indicating a number of attributes that they've been rated on (fictitious working example here). The data frame looks ...
• 101
4 views

time series clustering, necessary stationarity or not?

I am having a time series dataset for clustering purposes. How should non stationarity existing in my dataset to affect my cluster formation?
10 views

How to calculate distance for symmetric binary and nomianl variables?

In the existing function dist(), the only method for nominal variable is 'binary', and it's for asymmetric binary. However, I ...
14 views

Validate different unsupervised learning algorithms and by combination of parameters

I am applying various clustering algorithms on a given data set: CH: Calinski-Harabasz DB: Davies-Bouldin For each algorithm, I first want to obtain the best combination of parameters: ...
• 103
12 views

Retrive image from from features represented by histograms of oriented gradients

I am using histogram of oriented gradients for image classification using clustering in scikit learn. I am using hog from scikit-image to generate hog from 512x512 grayscale image. Here is an example: ...
10 views

Calculating new centroids when the centroids are chosen at random

When given two random points which are not instances in the dataset should I include the centroids in my calculations for the new centroids? For example in this link they are using the starting ...
• 1
7 views

How do you extract speerate structures from a cluster of points in 2d cordinate

I have a bunch points in x,y that correspond to so physical processes. My goal to extract and group points based on the event/process the correspond to. The image attached shows a example of how the ...
• 1
1 vote
25 views

Interpreting cluster variables - raw vs scaled

I already referred these posts here and here. I also posted here but since there is no response, am posting here. Currently, I am working on customer segmentation using their purchase data. So, my ...
• 2,215
15 views

How to divide earth into polygons based on a collection of labeled coordinates?

I have around one million labeled coordinates(latitude, longitude) all around the world, with around 10,000 unique labels(location_id). Each point corresponds to exactly one class(location_id). Each ...
• 1
15 views

matrix profile distance measure characterization

If there are various types of distances measures for time series, such as Euclidean, DTW, and shape-based ones, how can we characterize the matrix profile distance measure? Profiling one?
32 views

Find how the properties of an entity affect a certain property of its surrounding

We have a set of things (physical entities): ($A_1$, $A_2$, $A_3$, $A_4$,...). Each of those has certain attributes that we can measure at time $t = 0$. Let $B_i$ represent those attributes, so each ...
1 vote
45 views

Time Series Clustering on sales data -- any ideas?

I have a retail store dataset, and I am interested to do some time series clustering on this data, what idea you find interesting for this purpose? I have so far: What sales trends there are across ...
141 views

Why Do a Set of 3 Clusters Across 1 Dimension and a Set of 3 Clusters Across 2 Dimensions Form 9 Apparent Clusters in 3 Dimensions?

I am sorry if this is a well-known phenomenon but I can't quite wrap my head around this. I have a related question: How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions ...
1 vote
35 views

Accuracy drops when adding a fully connected layer for dimensionality reduction to a ResNet50

I'm training a ResNet50 for image classification and I'm interested in decreasing the dimensionality of the embedded layer, in order to apply some clustering techniques. The suggested dimension is ...
• 61
1 vote
11 views

What clustering algorithm is best for dataset with only binary categorical features

I have a dataset with a lot of binary categorical features and a single continuous target value. I would like to cluster them but I am not quite sure what to use? I have in the past used dbscan for ...
• 123
1 vote
25 views

High Performance Classification or Similarity Algorithim for Mixed Data Types?

I have a database holding 10-ish features that describe different breeds of dogs. They are mostly categorical features, but some provide ranges for values. Here's a demo representation of the database,...
10 views

Big difference between Bootstrap Values and Approximately Unbiased p-values

I'm clustering objects over many different descriptors. I chose a hierarchical clustering method (specifically average linking algorithm with euclidean distances) because I wanted to use bootstrap ...
6 views

Recommender system based on clusters

I'm wondering if this is a correct approach to build recommender systems: My problem: Recommend phone devices, you have device X and you are likely to switch to device Y. Understand the data. I want ...
10 views

Clustering Using SOM Codebook

I've recently been using the aweSOM R package for cluster visualisation, https://cran.r-project.org/web/packages/aweSOM/vignettes/aweSOM.html. In particular, the aweSOM package entails using ...
• 193
19 views

K-Means R vs K-Means Python different cluster values generating different bar Graphs

Below are 2 sets of code that do the same thing one in Python the other in R. They both graph the Kmeans the same with respect to PCA but once I do the bar chart at the end using the cluster Center ...
• 101
1 vote
25 views

Can clustering results based on probability be used for supervised learning?

I'm a beginner and I have a question. Can clustering results based on probability be used for supervised learning? Manufacturing data with 80000 rows. It is not labeled, but there is information that ...
• 11
15 views

Clustering with maximum weight and distance conditions

I have a set of weighted 2D points (coordinates x, y and weight w for each sample). I want to cluster these samples using minimum number of clusters, with the following conditions: Use the least ...
28 views

Does t-SNE have to result in clear clusters / structures?

I have a data set which, no matter how I tune t-SNE, won't end in clearly separate clusters or even patterns and structures. Ultimately, it results in arbitrary distributed data points all over the ...
• 480
1 vote
126 views

How To Develop Cluster Models Where the Clusters Occur Along Subsets of Dimensions in Multidimensional Data?

I have been exploring clustering algorithms (K-Means, K-Medoids, Ward Agglomerative, Gaussian Mixture Modeling, BIRCH, DBSCAN, OPTICS, Common Nearest-Neighbour Clustering) with multidimensional data. ...
8 views

Clustering finite size points

I'm looking for a clustering algorithm for a set of points that also have a radius (aka circles). Effectively I have a set of circles each with parameters (x, y, r). Just taking the circle's centre is ...
33 views

Hard time finding literature on feature clustering using Principal Component Analysis

Im new to StackExchange, so i am sorry if this is not the right way to ask a question on StackExhange. For my thesis I wish to propose a methode for future research on using PCA to cluster features (...
27 views

DB-Scan with ring like data

I've been using the DBScan implementation of python from sklearn.cluster. The problem is, that I'm working with 360° lidar data which means, that my data is a ring ...
• 111
1 vote
20 views

Is it possible use cluster analysis on word co-occurrences?

Problem: I am unsure if there is an appropriate clustering method to do the following: I wish to group a list of word co-occurrences into their possible clusters. Context: I have a dataset containing (...
11 views

best way to convert labels

I need to convert label Win,Lose,Draw to continous number, thich is the best method to do it? Win=1 , Draw=2, Lose=0 ?
11 views

key generation from feature vectors in high dimentions

I welcome any suggestions to solve the following hard problem: I have a dataset of float feature vectors of size 512 where each feature vector is extracted from a face image. I want to generate a key ...
29 views

Optimal clusters for K-means not clear - any ideas?

I have a toy dataset of 10,000 strings of people's names, addresses and birthdays. As a quirk of the data collection process it is highly likely there are duplicate people caused by typos and I am ...
• 187
8 views

Clustering when data is represented by multiple functional forms, all at once

I wish to cluster similar data where I have a collection of many $y$ vs $x$ data. A third variable $z$ also exists, but z doesn't always affect $y$ whereas $x$ always affects $z$. In terms of $x$ and \$...
13 views

Evaluate Dendrogram Statistical Significance

I have N=21 objects and each one has about 80 possible not NaN descriptors. I carried out a hierarchical clustering on the objects and I obtained this dendrogram. I want some kind of 'confidence' ...
1 vote
13 views

Is HDBSCAN a agglomerative hierarchical clustering?

I am looking at HDBSCAN and wondering whether it is Divisive or Agglomerative? I understand the two approaches, but I cannot seem to grasp which HDBSCAN utilises. Looking for some elaboration. https://...
28 views

Association between categorical variables with no hierarchy in Python

I have a dataset with over 100 possible variable occurrences across 20 columns. At first glance this problem seemed to fit into hierarchical clustering. I started testing with Agglomerative Clustering,...
• 21
11 views

Mixed Data Type Classification / Neighbor Algorithm

Here is a hypothetical simplified dataframe of my problem, which would be low dimensional (20ish features), containing some made-up information about certain dog breeds: Breed Min_Weight Max_Weight ...
48 views

Kmeans clustering in python - Giving original labels to predicted clusters

I have a dataset with 7 labels in the target variable. X = data.drop('target', axis=1) Y = data['target'] Y.unique() array(['Normal_Weight', 'Overweight_Level_I', '...
1 vote