Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
1
vote
1answer
8 views

Classification of multidimensional data to multidimensional clusters with a varying subcluster structure

I have a large dataset with mixed (numerical, categorical, textual) data that I need to classify. The clusters are well-defined, but multidimensional (i.e. vector-valued) and have a varying structure ...
0
votes
0answers
19 views

Three different errors using external information. Which one makes sense? (Or how to interpret each?)

My goal is to compare clustering methods considering different method and different number of clusters using an external information. Could anyone please give some opinion/ recommend book/paper about ...
-1
votes
1answer
11 views

How to group clusters with semantic similarity?

I have a list of job titles. I found the semantic similarity between them by using word2vec in spacy. Now I want job titles ...
2
votes
1answer
17 views

How to cluster texts by most relevant words

I have a huge amount of documents and every document has its own portrait, where a portrait has this structure (document_id, word, weight). TFIDF, basically. I want to cluster these documents into ...
1
vote
0answers
17 views

What is the best way to cluster this kind of data?

I have data that looks like this: The chart on the left is the trend and the smaller chart on the right is the box plot showing the distribution of means. Each color is the output of a particular ...
0
votes
0answers
11 views

Discrete Wavelet Transform Time Series

My problem is to cluster some time series together. But due to a huge length I was interested in using some methods to reduce the dimensionality. I was thinking of Discrete Wavelet Transform since the ...
0
votes
0answers
25 views

Considerations to take into account when clustering

My idea is to use clustering to perform stock segmentation based on risk, building different risk levels that might adapt better to different kind of users. Hence I have computed different risk ...
0
votes
0answers
20 views

How to assign the remaining points to clusters [closed]

Very recently I was asked to find a way to assign new points on clusters formed with data in the past. I have two different ideas: Compute the similarity of these data points with the centers of ...
1
vote
0answers
9 views

How can the labels of AgglomerativeClustering be re-computed?

I'm using scikit learn's AgglomerativeClustering on a large data set. I want to modify the distance_threshold after the model has already been computed. Computing ...
2
votes
0answers
14 views

Methods of de-emphasizing some dimensions in a cluster analysis

I'm trying to understand how "weightings" on different dimensions in a cluster analysis might relate to the range of values along a given dimension in the dataset. DATA SET List of 1,000 to ...
0
votes
1answer
16 views

What is the difference between spiral, flame, aggregation data

What is the difference between spiral, flame, aggregation data? What are the names of the columns, or what are the columns indicate? For example, spiral is like to: ...
0
votes
0answers
31 views

Grouping data where each cluster has a similar sum of a variable?

I have a data set of postcode areas and their populations. I would like to cluster the data set in a way in which each cluster has roughly the same population of 80,000. Phrasing mathematically I'd ...
-1
votes
1answer
12 views

Detect time pattern in sequence of events

I have a time series with a timestamp and an associated event: Time Event 1 A 2 B 3 C T A I was wondering if there is a technique/method to figure out which events most often precede others in a ...
0
votes
1answer
9 views

Unsupervised Hierarchical Agglomerative Clustering

I've read a number of papers where the authors talk about "Unsupervised Hierarchical Agglomerative Clustering". They seem to imply that the algorithm determines the number of clusters based ...
0
votes
0answers
6 views

Calculate Variance from Ward distances

The Ward method for clustering minimizes the total within-cluster variance. So I suppose that there is a link between the Ward distances that I got with the linkage function, and the variance of the ...
0
votes
1answer
36 views

how to cluster inseparable data

Suppose I have a dataset containing two very similar classes of data. By similar, I mean that the 'distance' between these two classes is very small. For example, one instance in Class 1 is the sum of ...
0
votes
0answers
16 views

Feature scaling for clustering

I want to cluster groups, using K-Means, DBSCAN, etc. algorithms, based on lat-lng coordinates along with other features such as dummy variables, continues variables (in different units). What would ...
0
votes
0answers
7 views

Clustering of longitudinal user generated data; determine at what point in time does the user “become” the final clustering outcome

I analyse a lot of telephone call log data sets (akin to user generated data) and I use k-means clustering a lot to look at the types of callers that exist in the data set. The callers are clustered ...
-1
votes
1answer
22 views

Machine learning: inferring parents from child data

In producing SCADA systems, we get large lists of signals. The signals can be a couple of thousands and all have verbal descriptions such as: ...
-1
votes
1answer
36 views

Can clustering be applied on Linearly distributed data?

Let's say I have a sample dataset df generated as: ...
0
votes
0answers
18 views

Clustering together words that appear together while down weighting words that appear too often

I was wondering if I could get some help finding a good model for the problem I have. I have a data set where each observation is a set of words that go together. So for example, it could be: ...
0
votes
0answers
25 views

Determining the optimal number of cluster and clustering method in hierarchical clustering

I'm working with some patient data and decided to cluster the patients. I came up with a distance metric, created a distance matrix, and performed hierarchical clustering. Now the question is, what ...
1
vote
0answers
12 views

Comparing Clustering Over Time

I've recently conducted a k-prototypes R routine on some mixed data. In particular, the data is health data concerning a certain public health intervention, with categorical variables for health ...
1
vote
0answers
14 views

Distributed Tensorflow VS Tensorflow on Spark

I am new to Deep Learning and Tensorflow. I learned TensorFlow can be code to run in stand-alone mode and in distributed mode. I also came across Tensorflow on Spark framework that will allow the ...
1
vote
0answers
9 views

Scoring samples after clusterings [closed]

I want to assign a score to all points in a group that I cluster several time. I want the score to indicate how much this point is grouped with the same individuals all time. I suppose this idea ...
0
votes
1answer
17 views

How to run unmodified Python program on GPU servers with scheduled GPUs?

Say I have one server with 10 GPUs. I have a python program which detects available GPU and use all of them. I have a couple of users who will run python (Machine learning or data mining) programs and ...
2
votes
0answers
45 views

Clustering large set of images

I've got some big datasets of images (a few million each), and I would like to cluster them according to images' visual similarities. I've extracted a feature vector for each image; the space of ...
-1
votes
0answers
24 views

how we can calculate Davies–Bouldin index (DB-index) for DBSCAN clustering?

I know that DB-index is not a perfect validity measure for evaluating the DBSCAN clustering method. But I see in some papers that this index is calculated for DBSCAN as an internal validity measure. ...
1
vote
1answer
20 views

Clustering using both text and numerical features

I have a dataset that contains 2 types of features, one is generated from doc2vec and one is numerical feature. I would like to perform clustering analysis on them. However, due to the size of doc2vec ...
0
votes
0answers
16 views

Finding clusters of 3D points where not all the points belong to a cluster

In my research, I'm trying to find clusters of brain activity, like in the following image: Where the red dots represent the origin of the activity (where the green pole is the direction). So ...
0
votes
0answers
5 views

How to cluster large time series?

I want to perform clustering on a dataset which has a few thousand stock prices time series, each one with daily prices for previous 5 years (a couple thousand days). What would be the best approach ...
0
votes
0answers
16 views

What are some good techniques to decrease the size of Image Embeddings returned by CNN model?

I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
0
votes
0answers
16 views

How to get the collective score from different test methods

im very new to this - I have a dataset of some patients with certain symptoms of sleep apnea. They were assessed for the risk of Sleep Apnea using ESS, BQ, SBQ and SACS (different testing ...
2
votes
3answers
42 views

Is it possible to change the input columns of a trained ML model while making predictions from it without affecting the accuracy?

Consider the following scenario. I have trained a K-Means model on some input features, say, (A, B, C, D and E). Now at the time of making predictions I want to make the model predict using only fewer ...
2
votes
1answer
28 views

Percentage of smaller dataset with respect to bigger dataset

I have two datasets, which are lists of multidimensional real-valued vectors. One dataset (call it $A=\{x_1, x_2, x_3, ..., x_n\}$ is of a big size, the other (call it $B=\{x_1, x_2, x_3, ..., x_m\}$)....
0
votes
1answer
23 views

Clusterize item set with items as vectors of features

I have to clusterize this dataset in which I have houses and water consumption in this form: $$ House1 = (x_{1},x_{2}... x_{n});\\ House2 = (y_{1},y_{2}... y_{n});\\ House3 = (z_{1},z_{2}... z_{n});\\ ...
0
votes
0answers
26 views

Comparison between DBSCAN and single linkage hierarchical clustering

I am studying clustering algorithms and I want to find a good example where single link hierarchical clustering algorithm returns better cluster results than DBSCAN (having provided valid parameters). ...
1
vote
1answer
31 views

group similar subjects and train only using them

I have a dataset with 5k subjects. It's a binary classification problem where I have 3000 positive and 2000 negative subjects. Now to build a model, I don't like to train the usual way (where we build ...
1
vote
1answer
21 views

Which data sets would help to predict (exponential) market trends?

Which kind of datasets do websites such as MeetGlimpse, trends.co, explodingtopics.com use to detect exponential market trends? I love them (not affiliated) and would like to better understand how ...
1
vote
1answer
22 views

How to aasign a new data point to a cluster?

I have a userdataset which contains fields like ['age','gender', 'computer_literacy', 'vision', 'colour_blind', 'education', 'font_size','colour']. I clustered these data and assigned the new cluster ...
0
votes
0answers
10 views

How do we use a Hierarchical clustering model with DTW again?

I've been trying to cluster time series of shape (1, 400), so 1 row and 400 columns which correspond to 400 timesteps. My train set is of size (1000, 400) so 1000 time series. I have calculated a ...
0
votes
0answers
56 views

Implementing Gaussian Mixture Model from scratch on an Image dataset

I recently learned about GMM. From what I understand it basically gives a probability that a given data point belongs to a specific cluster. I need to implement GMM on the STL10 training dataset with ...
0
votes
0answers
10 views

Evaluating the performance of tracking multiple objects detected with object detection

I have a ground truth dataset where the objects have been manually annotated and each object have been provided an ID that is consistent through time. There are no false positives or false negatives ...
0
votes
0answers
19 views

What's the best way to detect crowds?

I have a dictionary containing people and the distance between each pair in the following format: ...
1
vote
1answer
47 views

How to interpret results of a Clustering Heart Failure Dataset?

I am doing an analysis about this dataset: click In this dataset there are 13 features, 12 of input and 1 is the target variable, called "DEATH_EVENT". I tried to predict the survival of the ...
2
votes
1answer
31 views

Why the Silhouette Score and optimal number of Cluster changes when using 2D and 3D data?

I am experimenting with Kmeans clustering. My data (vectors) was in 300 dimensions which I am converting into 2D and 3D using PCA. Now, to find the optimal number of clusters, I used the Silhouette ...
1
vote
1answer
22 views

Grouping by similarity

I would like to find a way/algorithm to group people into, say, four groups by their answer similarity to yes/no questions. So, each pair of people in one group would have given the same answers for a ...
0
votes
0answers
16 views

Efficient way to cluster millions of face embeddings

I'm currently working on a face clustering system that gets incremental new input data. I use the 128d face embeddings given by FaceNet. I already tested these Algorithms: Chinese Whispers, DBSCAN and ...
2
votes
1answer
23 views

Recommender/Clustering data to support a hypothesis. Is this a valid use-case for unsupervised ML?

I have a dataset where some items have been labelled (categorized into 4 classes [A,B,C,D]). However, there is a vast majority of the dataset which has not been labelled. My hypothesis is that there ...
1
vote
0answers
13 views

Error: cannot allocate vector of size 20.0 Gb [closed]

I'm running kmeans in RStudio (Version 1.3.1093): km.res <- eclust(df, "kmeans", k = 3, nstart = 25, graph = FALSE) But I keep getting this error message: "cannot allocate vector of ...

1
2 3 4 5
22