Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

351 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
3answers
215 views

Cluster evolution over time

I have a dataset of transactional data with customer ID and I want to segment the dataset into groups using cluster analysis. I'm interested in following the evolution of each cluster over time, but ...
4
votes
1answer
453 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
4
votes
2answers
100 views

KMeans clusterization on documents

Whether correct or not, I'm not able to judge being myself in the early days of the Data Science. However, I have applied a Kmeans on a corpus where some random documents (very short sentences) have ...
4
votes
0answers
95 views

Fixed-radius range search in non-Euclidean space

I'm trying to find an indexing data structure most suitable for my metric space: set of IP network related data (IP addresses, ports, TCP flags, ...), distance function is continuous, non-Euclidean ...
3
votes
1answer
84 views

What Clustering Method Should I Use?

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight. I need to finalize a bayesian ...
3
votes
1answer
80 views

Sentiment analysis of tweets (Train model on a labelled dataset and use on some other unlabelled data)

I have a huge amount of tweets on a particular topic say 'ABC' and the data is not labelled. I want to perform multi-class sentiment analysis of these tweets. I tried many unsupervised clustering ...
3
votes
1answer
122 views

PCA and k-means for categorical variables?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
3
votes
0answers
23 views

Visualizing the difference of a set of strings

I have a distance metric on a collection of strings on the order of tens of thousands. What would be an intuitive way to summarize how 'different' these strings are or when they overlap? My goal is, ...
3
votes
2answers
50 views

find most dense neighborhood of points in high dimensional space

I'm working on a project where I have many high-dimensional points and I want to find the most dense neighborhood of them. Ideally, out of my ~500 points that are each a 4x300 matrix (300 ms time ...
3
votes
1answer
281 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
3
votes
2answers
791 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
3
votes
1answer
29 views

Identify members who are likely to switch where they receive drug administration

I have access to medical claim data from a large health insurance company. As some of you may know there is a large delta between the price of drug X depending on where it is administered. My company ...
3
votes
0answers
63 views

How to remove noise using morphological filtering

I have two groups of dots that both contain noise between them: The line that separates the two groups in the picture is diagonal in shape. I tried to use morphological filtering on this image to ...
3
votes
2answers
1k views

K-modes implementation in pyspark

I'm looking for an implementation of k-modes in pyspark. I found this and this as implementations. First, I tried implementing k-modes using the first link and faced issues. So I went ahead and tried ...
3
votes
2answers
249 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
3
votes
3answers
771 views

clustering 2-dimensional euclidean vectors - appropriate dissimilarity measure

I've got a set of approx. 50 000 2-dimensional euclidean vectors which are connected with 20 groups, i.e. each group has approx. 2500 2-dimensional euclidean vectors. My data includes endpoints ...
3
votes
0answers
161 views

Clustering with Replicator Neural Network

I'm trying to cluster an unknown set of data with a replicator neural network. The number of clusters is determined by the number of neuron units in the middle layer, multiplied by the number of steps ...
3
votes
1answer
120 views

Finding dominating attributes with in the clusters generated

I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
2
votes
0answers
29 views

Clustering and getting threshold to classify data points

I have real world data set (120 data points) about enterprises, containing 4 features. I would like to put these enterprises in exactly 4 categories based on the values of these specific features (an ...
2
votes
1answer
32 views

How can I reduce the number of dimensions using a Clustering algorithm in a mixed dataset?

I am working with a mixed data set, corresponding to TV consumption data, with the aim of reducing the number of features to only those relevant to detect TV consumption patterns (or consumption ...
2
votes
0answers
17 views

R: How to find the rows in the input files that correlate to the 2 “populations” in the plot

I am completely new to clustering analysis. Let's say I have a file of the format: ...
2
votes
1answer
27 views

How to cluster texts by most relevant words

I have a huge amount of documents and every document has its own portrait, where a portrait has this structure (document_id, word, weight). TFIDF, basically. I want to cluster these documents into ...
2
votes
0answers
15 views

Methods of de-emphasizing some dimensions in a cluster analysis

I'm trying to understand how "weightings" on different dimensions in a cluster analysis might relate to the range of values along a given dimension in the dataset. DATA SET List of 1,000 to ...
2
votes
1answer
21 views

Comparing Clustering Over Time

I've recently conducted a k-prototypes R routine on some mixed data. In particular, the data is health data concerning a certain public health intervention, with categorical variables for health ...
2
votes
0answers
49 views

Clustering large set of images

I've got some big datasets of images (a few million each), and I would like to cluster them according to images' visual similarities. I've extracted a feature vector for each image; the space of ...
2
votes
1answer
421 views

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: ...
2
votes
0answers
19 views

Resource-unintensive (low complexity) methods for large-scale unsupervised clustering?

I'm working on an issue where I need to cluster user types on a scale in an unsupervised manner. I've been looking at the basics like KNN and K-means etc., but I found it hard to scale, as these ...
2
votes
0answers
20 views

Deep Continious Clustering algorithm - just one output cluster

I use the DCC algorithm to cluster some data. The whole algorithm is available here, but shortly it is: construct mkNN graph of the data points (the connected components of it are the clusters). ...
2
votes
0answers
54 views

Clustering similar sequences using hidden markov model

I have several sequences of different lengths. For example, ...
2
votes
0answers
63 views

Classification and clustering of Time series data of temperature

I have a time series recorded data of temperature. This is what my data looks like: The change in data represents specific event or a class which I would like to detect when new incoming data. ...
2
votes
2answers
70 views

Using KNN to categorise inventory (physical stock items) - is it the best way?

I'm working on a machine learning problem involving inventory (i.e. physical retail stock), however through the cleaning (outlier removal) process some of the items (via their corresponding ...
2
votes
0answers
16 views

Identifying persistent clusters within a series of graphs

The task is to identify persistent clusters, i.e., groups of nodes that "persist" as clusters (tend to form a cluster) in a series of graphs. This is how I approached the problem: I form a ...
2
votes
1answer
274 views

Clustering mixed data types - numeric, categorical, arrays, and text

I have a dataset with 4 types of data columns: ...
2
votes
1answer
201 views

How to get the probability/closeness of a sample belonging to a specific cluster?

I'm new to this so please let me know if my logic of comparing cosine similarity and k-means is incorrect I got a set of ...
2
votes
2answers
263 views

Techniques for Cluster Analysis of a Very Large (n=140000) Binary Dataset in Python?

In essence: what techniques in Python are possible to find clusters/trends in a very large categorical dataset? My very large dataset (140000 rows/observations, 80 variables) of categorical data has ...
2
votes
2answers
46 views

How to decide who to market? Clustering or Decision Tree?

I am working with a dataset that has enough observations and ~ 10 variables, half of the variables are numeric another half of the variables are categorical with 2-3 levels (demographics) one ID ...
2
votes
0answers
270 views

K-Means Clustering Profile Plot & Data Normalization

I am new to k-means clustering and I am working on a project on cryptoanalysis. I have a few questions and I hope to get some help here. I have four variables and my variables data values can range ...
2
votes
0answers
20 views

Assigning points to fitted planes

I’m working on a project involving fitting planes to 3D point clouds. The actual plane fitting part is working fine, but I’m trying to decide the best way to actually bound the fitted planes by the ...
2
votes
1answer
211 views

Customer Segmentation and Category association

I have to solve two questions on the following dataset: 1. arrange customers into mutually exclusive groups.explain the clusters. 2.identify 1-1 product category association rules for each cluster, i....
2
votes
0answers
179 views

Will flattening multivariate time series data before clustering make the results meaningless?

I have a large number of financial time series that I wish to do cluster analysis on. Each time series has the same length and spans multiple years of daily data (returns, volatility, etc.). As part ...
2
votes
0answers
68 views

News de duplication dataset

I am looking for a news dataset with semantically duplicate news articles tagged. Basically all the news articles which talk about the same story should be grouped. The stories can be worded ...
2
votes
1answer
31 views

How to retrain a K-Modes model based on daily data?

I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
2
votes
1answer
42 views

Clustering Weekday Weekend Data and Multicollinearity

Hi I have data of weekday and weekend step counts in which I extracted metrics from them such as the wd steps, we steps, standard deviation of wd steps, standard deviation of we steps and so on... <...
2
votes
1answer
37 views

What's the good index to choose number of clusters so that obtained clusters are homogeneous?

I perform a clustering on one-dimensional dataset and I need a way to automatically decide what's the optimal number of clusters from $k \in \{2, 3, 4, 5, 6\}$. The number of observations to cluster ...
2
votes
0answers
195 views

How to tune / choose the preference parameter of AffinityPropagation?

I have large dictionary of "pairwise similarity matrixes" that would look like the following: similarity['group1']: ...
2
votes
2answers
46 views

Identifying common keyphrase frequency in large dataset

I have a dataset of profiles that contain freeform text describing the work history of a number of individuals. I would like to attempt to identify frequently used words or groups of words across the ...
2
votes
1answer
47 views

Find shared properties of a cluster samples

I have a dataset which contains ~15 features. With the elbow method, I found out that the optimal number of clusters is probably four. Therefore, I applied the K-means algorithm with four clusters. ...
2
votes
1answer
33 views

Sensorfusion: Generate virtual sensor based on analysis of sensorsdata

I have a steam engine which is equipped with the following sensors: temperature sensor in the boiler room temperature sensor in the heating room pressure sensor in the boiler room rotations-per-...
2
votes
1answer
147 views

Clustering time series based on monotonic similarity

Context I am involved in a task of clustering 1500 time series of 500 observations into a few number of clusters. The time series share all the same observed property at different spatial locations, ...
2
votes
0answers
34 views

Measure of variety within list/cluster

I have a dataset of about 53000 points. It has been clustered twice, based on two sets of unrelated attributes. For the first clustering (clustering 1) I used DBScan, and it ended up with about 700 ...

1
2 3 4 5
8