People who code: we want your input. Take the Survey

Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

What are some good techniques to decrease the size of Image Embeddings returned by CNN model?

I want to extract features from pre trained ResNet model for over 2M data. Problem? Even with the average pooling applied on the last layer's result, it provides a ...
0
votes
0answers
17 views

How to get the collective score from different test methods

im very new to this - I have a dataset of some patients with certain symptoms of sleep apnea. They were assessed for the risk of Sleep Apnea using ESS, BQ, SBQ and SACS (different testing ...
2
votes
3answers
48 views

Is it possible to change the input columns of a trained ML model while making predictions from it without affecting the accuracy?

Consider the following scenario. I have trained a K-Means model on some input features, say, (A, B, C, D and E). Now at the time of making predictions I want to make the model predict using only fewer ...
2
votes
1answer
29 views

Percentage of smaller dataset with respect to bigger dataset

I have two datasets, which are lists of multidimensional real-valued vectors. One dataset (call it $A=\{x_1, x_2, x_3, ..., x_n\}$ is of a big size, the other (call it $B=\{x_1, x_2, x_3, ..., x_m\}$)....
0
votes
1answer
24 views

Clusterize item set with items as vectors of features

I have to clusterize this dataset in which I have houses and water consumption in this form: $$ House1 = (x_{1},x_{2}... x_{n});\\ House2 = (y_{1},y_{2}... y_{n});\\ House3 = (z_{1},z_{2}... z_{n});\\ ...
0
votes
0answers
76 views

Comparison between DBSCAN and single linkage hierarchical clustering

I am studying clustering algorithms and I want to find a good example where single link hierarchical clustering algorithm returns better cluster results than DBSCAN (having provided valid parameters). ...
1
vote
1answer
37 views

group similar subjects and train only using them

I have a dataset with 5k subjects. It's a binary classification problem where I have 3000 positive and 2000 negative subjects. Now to build a model, I don't like to train the usual way (where we build ...
1
vote
1answer
22 views

Which data sets would help to predict (exponential) market trends?

Which kind of datasets do websites such as MeetGlimpse, trends.co, explodingtopics.com use to detect exponential market trends? I love them (not affiliated) and would like to better understand how ...
1
vote
1answer
26 views

How to aasign a new data point to a cluster?

I have a userdataset which contains fields like ['age','gender', 'computer_literacy', 'vision', 'colour_blind', 'education', 'font_size','colour']. I clustered these data and assigned the new cluster ...
0
votes
0answers
15 views

How do we use a Hierarchical clustering model with DTW again?

I've been trying to cluster time series of shape (1, 400), so 1 row and 400 columns which correspond to 400 timesteps. My train set is of size (1000, 400) so 1000 time series. I have calculated a ...
0
votes
0answers
213 views

Implementing Gaussian Mixture Model from scratch on an Image dataset

I recently learned about GMM. From what I understand it basically gives a probability that a given data point belongs to a specific cluster. I need to implement GMM on the STL10 training dataset with ...
0
votes
0answers
10 views

Evaluating the performance of tracking multiple objects detected with object detection

I have a ground truth dataset where the objects have been manually annotated and each object have been provided an ID that is consistent through time. There are no false positives or false negatives ...
0
votes
0answers
19 views

What's the best way to detect crowds?

I have a dictionary containing people and the distance between each pair in the following format: ...
1
vote
1answer
69 views

How to interpret results of a Clustering Heart Failure Dataset?

I am doing an analysis about this dataset: click In this dataset there are 13 features, 12 of input and 1 is the target variable, called "DEATH_EVENT". I tried to predict the survival of the ...
1
vote
1answer
136 views

Why the Silhouette Score and optimal number of Cluster changes when using 2D and 3D data?

I am experimenting with Kmeans clustering. My data (vectors) was in 300 dimensions which I am converting into 2D and 3D using PCA. Now, to find the optimal number of clusters, I used the Silhouette ...
1
vote
1answer
23 views

Grouping by similarity

I would like to find a way/algorithm to group people into, say, four groups by their answer similarity to yes/no questions. So, each pair of people in one group would have given the same answers for a ...
0
votes
0answers
35 views

Efficient way to cluster millions of face embeddings

I'm currently working on a face clustering system that gets incremental new input data. I use the 128d face embeddings given by FaceNet. I already tested these Algorithms: Chinese Whispers, DBSCAN and ...
2
votes
1answer
25 views

Recommender/Clustering data to support a hypothesis. Is this a valid use-case for unsupervised ML?

I have a dataset where some items have been labelled (categorized into 4 classes [A,B,C,D]). However, there is a vast majority of the dataset which has not been labelled. My hypothesis is that there ...
1
vote
1answer
40 views

How to compare topics generated from topic modeling from different datasets?

I have two datasets of a similar theme. Let's assume Dataset A and Dataset B. Using the top2vec model (https://github.com/ddangelov/Top2Vec) (https://arxiv.org/abs/2008.09470) on each dataset, I came ...
5
votes
3answers
175 views

What's the difference between data classification and clustering (from a Data point of view)

What are the differences and the similarities between data classification (using dedicated distance-based methods) and data clustering (which has certain defined methods such as ...
2
votes
1answer
396 views

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: ...
1
vote
1answer
25 views

Is knn similar to this version of k-means?

If we use k-means in a dataset where k is equal to the number of points in the dataset, and each cluster is made out of only a point. Considering that we have given a distance method, we can classify ...
1
vote
1answer
12 views

Can we define a data partitioning in K clusters, by cutting the branches of the tree at some levels in the tree below the root node?

Assume we have a dendogram (hierarchical clusterisation tree), can we define a data partitioning in K clusters, by cutting the branches of the tree at some levels in the tree below the root node?
0
votes
0answers
30 views

Agglomerative Hierarchical Clustering on Images

My goal is to implement the agglomerative hierarchical clustering algorithm on an RGB image to cluster every pixel until some stopping criteria is reached. In order to do so, I assumed that each pixel ...
0
votes
1answer
42 views

Algorithm to find Unique users from their transactions

Scenario An eCommerce site allow users to purchase items without creating an Account. However, it captures following attributes from the Purchases based on the type of payment and type of delivery: ...
1
vote
1answer
27 views

Show distribution of users affected by outlier response times

My dataset is performance metrics (response time) of a web page over the course of a single day. The data looks roughly like this: ...
0
votes
1answer
97 views

how to compare between kmeans and hierarchical clustering results

I am using 2 types of clustering algorithm I apply hierarchical clustering the K-means clustering using python sklearn library Now the results are a little bit different so how can I compare the ...
0
votes
1answer
41 views

How to use scikit-learn to extract features from text when I only have positive and unlabeled data?

I'm looking for something similar to this https://scikit-learn.org/stable/auto_examples/text/plot_document_classification_20newsgroups.html#sphx-glr-auto-examples-text-plot-document-classification-...
1
vote
1answer
17 views

Document clustering to merge common labels

I am building a recommendation system and I have to clean up some of the labels that I have. For example of the data df['resolution_modified'].value_counts() Gives ...
1
vote
1answer
151 views

What is meta- data and meta features?

I want to know what is metadata and what is meant by meta features? When I google Meta Features what I get is feature selection tool called "Meta-Feature". What is the function of feature ...
0
votes
0answers
34 views

Convert TFIDF Values to Vector Space Model

I'm working on a project using tf-idf values and cosine similarity for clustering. As my database (elasticsearch) provides tfidf values out of the box (term_freq & doc_freq), my code involves ...
0
votes
0answers
16 views

What does MiniBatchKMeans fit's reassignment_ratio parameter do exactly?

I am using scikit-learn MiniBatchKMeans to do text clustering. In the constructor method, there is a parameter reassignment_ratio, which is described in the ...
0
votes
1answer
115 views

How to interpret the sample_weight parameter in MiniBatchKMeans?

I am using scikit-learn MiniBatchKMeans to do text clustering. In the fit() function there is a parameter sample_weight described as follows: The weights for each ...
1
vote
2answers
35 views

Elasticsearch + Clustering

I'm currently working on a project that relies on the clustering of documents into an unknown number of clusters, based on a similarity threshold (ideally using cosine distance between tf-idf vectors)....
1
vote
1answer
43 views

Comparing TFIDF vectors of different shapes

I'm working on a project using TF-IDF vectors and agglomerative clustering -- the idea is that the corpus of documents increases over time, and when a new document is added, the mean cosine similarity ...
0
votes
1answer
50 views

Decision trees for anomaly detection

Problem From what I understand, a common method in anomaly detection consists in building a predictive model trained on non-anomalous training data, and perform anomaly detection using the error of ...
1
vote
1answer
37 views

Are LDA clusters identical across different runs?

for a given corpus are the Latent Dirichlet Allocation clusters for it is unique in general? How about the gensim multi-process implementation of LDA? are there ...
0
votes
0answers
16 views

Dimensionality Reduction for Function Fitting method using Kernels

I have a set of continuous noisy measurements $x_i \in R^n$ with $i=1,...,N$ for which I know the value range, i.e. $x_{min} \leq x_i \leq x_{max}$. Corresponding to the measurements, I have a set of ...
0
votes
0answers
20 views

Hierarchical clustering in R filtering variable

I would like to test the added value of features compared to currently used predictors. First, I checked if features were not correlated to the predictors (volume and intensity) I already use, and for ...
1
vote
2answers
57 views

Coloring clusters so that nearby clusters have different colors

I have clustered a large number of points (~3000) into (~400) clusters. I want to plot the data and visualize the clusters. I want to make sure that nearby clusters have different colors. Can ...
1
vote
1answer
26 views

Alterable Similarity Join for Time Series

I'm currently trying to implement the MPdist (matrix profile distance) algorithm for time-series data, but I've developed a new distance metric that I'd like to use in place of the Euclidean metric. I'...
1
vote
1answer
72 views

Visualising K-Means clusters for 3D data in R

I have an excel file that contains 485k rows x 3 columns of integer values. Sample data: ...
0
votes
1answer
50 views

K-means clustering to separate temperature vertical profiles

I have temperature measurements from weather stations in a mountainous region and I want to obtain a vertical profile from these data at any given time. In a simple case one can just plot all values ...
6
votes
2answers
79 views

Is sampling a valid way to reduce complexity?

I'm facing an issue where I have a massive amount of data that I need to cluster. As we know, clustering algorithms can have a very high O complexity, and I'm looking for ways to reduce the time my ...
0
votes
1answer
360 views

Clustering Tweet Data using DBSCAN Algorithm

I am doing a tweet clustering using DBSCAN algorithm. I use all the preprocessing steps and convert sentences to vector format before applying the algorithm. ...
1
vote
1answer
49 views

DBSCAN on textual and numerical columns

I have a dataset which has two columns: title price sentence1 12 sentence2 13 I have used doc2vec to convert the ...
1
vote
0answers
28 views

Building an unsupervised learning model to detect suspicious transactions using DBSCAN [closed]

I am working on building a unsupervised learning model to detect suspicious transactions using DBSCAN. Do I train the model on all data columns (columns like account number, transaction date, ...
0
votes
1answer
27 views

How to study the effect of eps in sklearn.cluster.DBSCAN?

I posted this question on stackoverflow.com and have not received any answer. In case I get an answer from one of them, I will inform on the other. I have a dataset and is requested by my professor ...
3
votes
2answers
106 views

Is there a clustering algorithm which accepts some clusters as input and outputs some more clusters?

Heres the task: I have data I don't know much about. The final task is to build a classifier to classify the samples into a few categories. Some of the categories are pretty clear, we can easily use ...
3
votes
1answer
55 views

Is there a clustering algorithm that works with only pairwise distances as input?

My data are places for which I know all pairwise travel times (='distances'), and I want to cluster those places minimising the total pairwise travel time inside a cluster. K-means can't be used ...

1 2
3
4 5
23