Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
10 views

Determining clusters of time series

I have been given time series of N stock prices and time series of K sectoral indices. (Sectoral index is index made from all stocks belonging to that sector.) I have not been given names of any of ...
  • 99
1 vote
1 answer
24 views

Do you have to use clustering with SciKit-Learn's Mutual Information metric?

I'd like to calculate the mutual information between two datasets, but I'd prefer not to cluster them first. I'm thinking of using SciKit-Learn's mutual_info_score ...
  • 523
0 votes
1 answer
19 views

How to cluster based on sensor data? - My first data science job

I'm on my first (real), data, programming job. As everyone can imagine, this can be quite hard and I learn a lot from it, given I am a data science student in university. However, I am completely ...
0 votes
0 answers
24 views

SequentialFeatureSelector and scoring for clustering

I want to use SequentialFeatureSelector from scikit-learn to do feature selection for clustering (K-Means). [Here][1] is a list of available evaluation scores for clustering. What I do not understand ...
0 votes
0 answers
10 views

Product scrape data clustering model

I am developing a tool that scrapes multiple websites for products. Some websites sell the products themselves, while others feature products sold by third-party sellers that may have incorrect ...
  • 101
0 votes
1 answer
21 views

Statistical test for comparing number of clusters in data

I am performing $K$-means clustering on a dataset consisting of $n$ observations and $d$ variables, and I'm trying to determine the optimal number of clusters. Is there a test that can determine the ...
0 votes
0 answers
6 views

getting wrong cosine similarity when using face_recognition.faceencodings

I am trying to calculate cosine similarity between two face encodings returned by face_Recognition.face encoding() which 128d vector which always is above 0.8 also is the face encoding normalized ...
1 vote
3 answers
63 views

What's the fastest clustering package in Python?

I'd like to perform clustering analysis on a dataset with 1,300 columns and 500,000 rows. I've seen that clustering algorithms are available in SciKit-Learn. But I'm worried that the algorithms will ...
  • 523
0 votes
1 answer
32 views

Patterns extraction in time serie with DTW

I have a long time serie, let's say 1000 items. I want to find patterns in it of different lengths from 10 to 100 elements. To do this, I extract sliding windows of different lengths and calculate ...
0 votes
1 answer
10 views

MEL VS linear spectrograms for bioacoustics machine learning

I don't have background in bioacoustics but working on a data-science project in bioacoustics. I am working with animal vocalizations recorded at sampling rate of 250000. Animals are bats, which are ...
0 votes
0 answers
19 views

Assign clusters to new data using a model based on previously gained data

In an experiment, we gained data through an online survey. We used this data to conduct a cluster analysis (using the library(mclust) library). According to the ...
  • 1
0 votes
1 answer
16 views

Scoring vectors by "distinctness"

I have a set of vectors in a vector space and I'm looking for some score that measures how "distinct" each vector is from the others. I suppose this is a well-known problem but I have no ...
0 votes
0 answers
11 views

Clustering methods for text and image features

I want to build a recommender system with unlabeled data and used TF-IDF to extract text features from a given short description and VGG-16 to extract image features. I am looking for a way to combine ...
  • 1
1 vote
1 answer
27 views

best algorithms for clustering customers, customer segmentation

I have a dataset mixture of categorical and numerical variable, I was wonder what are the best algorithms to cluster customers? how to find the underlying patterns that segments a customer??
0 votes
1 answer
30 views

Efficient ways of clustering for big data

I have a task which is customer segmentation with 120k users and a record of their purchases which is +3 million records of data, the approach I want to use is to use clustering algorithms like kmeans ...
1 vote
1 answer
21 views

Movement in cohorts

I am working on a user sales data which gets updated week over week. Based on the sales done in each week, the user is categorized in segment A, B or C. This means size of each segment could change ...
  • 31
0 votes
0 answers
19 views

Cluster-Based Anomaly Detection (and PCA)

I have a dataset of user operations (250 types of operations) in a trading platform and my task is to extract features, flagging rules, other insight for fraud prevention/anomaly detection, or some ...
0 votes
1 answer
14 views

Maximum categories for categorical variables in K-Means clustering

I am trying to perform K-means clustering on a dataset, and one of my categorical features has 96 possible options. Would this be too many features for one variable to have? The alternative would be ...
  • 101
0 votes
1 answer
12 views

How can I show data clustering after building a Convolutional Variational Autoencoder?

I am brand new to Machine Learning, and just followed Tensorflow Convolutional Variational Autoencoder Tutorial using my own dataset instead of the MNIST set. I was able to successfully display the 2D ...
0 votes
0 answers
7 views

Analysis to identify similar occupations by frequency of skills requested in job postings (in R)

I have access to a dataset of job postings, which for each posting has a unique posting ID, the job posting occupation, and a row for each skill requested in each job posting. The dataset looks a bit ...
  • 1
0 votes
1 answer
63 views

How to compare labels from clustering analysis and original ones?

I was asked to run a clustering analysis to assess the validity of labels for a manually labelled dataset. I can simply save the actual labels (4 classes: 0, 1, 2, 3) and run clustering analysis (let'...
0 votes
0 answers
14 views

Select Topic Words from Clusters

I am following this solution for clustering: https://towardsdatascience.com/clustering-contextual-embeddings-for-topic-model-1fb15c45b1bd For step four "Select Topic Words from Clusters", I ...
  • 129
1 vote
1 answer
26 views

Customer Segmentation with mixed data [closed]

I want to perform clustering. I am reading about this topic but I am totally confused. My dataset has 490 observations and it consists of numerical data (3 columns: Recency, Frequency, Monetary), ...
  • 11
0 votes
2 answers
29 views

NLP topic clustering

In my dataset, I have 500 abstracts. The goal is to cluster them in 2 topics. One topic must have those abstracts which contain some list of words or similar words and the rest of the abstracts in ...
0 votes
0 answers
27 views

How assign a new sentence embedding to a cluster in fast clustering

I am working on clustering sentence embeddings obtained from sentence transformer, for which I used fast clustering (clustered on cosine similarity). https://www.sbert.net/examples/applications/...
  • 13
0 votes
0 answers
17 views

Determine unusual occurrence of words in classes

I am working on a project where I have 20+ classes/groups. Each of these groups perform certain text searches. I am looking for specific keywords example 'code' which is an anomaly. The challenge is ...
1 vote
1 answer
19 views

Which pre-trained model to select to generate embeddings from shop names written in English?

Good afternoon! I have a dataset with thousands of shop names written in English. Several shop names might belong to one business entity, for instance, shops with names "KFC 001", "WWW....
  • 11
0 votes
1 answer
58 views

Cluster products that are frequently bought together

I have a dataset of articles metadata for each article, so something like this: product_id color type 1234 red t-shirt and another containing the transactions of customers, which looks like this: ...
  • 1
0 votes
1 answer
40 views

In DBSCAN, can the distance between a Noise Point and Border Point be less than Epsilon?

In DBSCAN: A core point is a point which has at least "MinPts" points inside its Epsilon radius. A border point is a point inside the Epsilon radius of a core point, but it has a number of ...
0 votes
0 answers
21 views

Ways to measure similarity between two datasets with different clusters

I have two datasets, each one with a different set of clusters of different shapes and sizes. What are some ways to compute the similarity of the clusters of these two datasets? (Note that both ...
1 vote
0 answers
13 views

How to do unsupervised clustering on sentences to find intents

I am working on chatbot for students. So, I have chatlogs on conversation between student and tutor, which is on mathematical problems (no labels). First thing I want to find is intents, such as ...
  • 13
2 votes
1 answer
34 views

Identifying competition climbing styles from World Cup data

In competitive climbing, it is well known that there are many different styles, differentiated by e.g. the steepness of the climbing wall. Some athletes excel at certain styles more than others. I ...
  • 121
0 votes
0 answers
14 views

Finding centroids with less difference but high variability

I have a data set including lat , lon and id's for some places. Also I have another table which has features for each id's. I wanted to find cluster centroids for those places. Using lat and lot ...
0 votes
1 answer
39 views

Cluster URLs based on their pattern

I am new to clustering techniques and I highly value any input you can provide for my problem bellow. Basically, I want to cluster URLs based on their structural patterns. for example cluster1 - ...
  • 101
0 votes
1 answer
26 views

Clustering on Raw Image Pixel Array

I have an array in the shape of (105, 105, 3). When I do plt.imshow(array) it outputs: How can I run a clustering algorithm directly on this image? Do I need to convert the pixels to cartesian ...
0 votes
0 answers
12 views

How to quickly know if there is a cluster or several clusters in a certain area?

In my research experiment, I want to use genetic algorithm to optimize a set of 2d points in an area. The objective is to make those points to be less average scattered. It means that there should one ...
0 votes
1 answer
19 views

How to isolate a clear relationship from a subset of data with lots of noise and outliers

I am doing an analysis of aircraft data and I want to see how much fuel is burnt on landing. There are 2 main factors aircraft type and landing time (ie. time elapsed) However there is a cheeky third ...
0 votes
1 answer
32 views

What can be the reasons for 95% of samples belong to one cluster when there is 5 clusters?

'''I used the k-means algorithm to clustering set of documents which are textual data only. The document has 2lack records. Surprisingly the result for the clustering is 90% of records is storing in 1 ...
0 votes
0 answers
10 views

What does near-cut (threshold) graph say about original complete weighted graph?

CROSSPOST: https://math.stackexchange.com/q/4588249/117548 ORIGINAL post, preamble Start from a complete graph with weighted edges (e.g. in $[0,1]$ interval). Continuously increasing threshold $t$ ...
  • 21
0 votes
0 answers
54 views

How to do DBSCAN clustering with mixed variables (numerical features and binary/ordinal variables)?

I have a question written at the end of the post which refers to the "Distances" paragraph. The other first two paragraphs give additional info. Context I'm working on a project where I have ...
5 votes
2 answers
416 views

Dynamic clustering in NLP dataset

Let's imagine this list: corpus = ['cat','banana','dog','horse','apple','tiger','snake'] I am looking for a way to build a vectorizer with dynamic clustering. I'm ...
0 votes
0 answers
11 views

How to graphic evolution of clusters in time

I am currently working in the definition of the profiles of patients of a specific disease. I have applied an unsupervised clustering technique to my dataset obtaining different patient profiles for ...
2 votes
1 answer
83 views

Incremental clustering algorithm

I am looking for an incremental clustering algorithm. By incremental I mean an algorithm that builds clusters starting from an initial dataset and that is able to progressively ingest new items/...
  • 121
0 votes
0 answers
19 views

Clustering mixed type variables with Orange

I wonder if with Orange it is possible to cluster mixed type data, so a dataset with numeric as well as discrete (categorical) data (ordered / unordered). Can you show an example of how that could be ...
0 votes
1 answer
15 views

How to identify new potential customers from existing customers profile?

I have firmographic data of all the possible customers. Data includes sales, profitability, capital, organisation size, geographic location, industry, etc. What is the best way of identifying new ...
  • 233
0 votes
0 answers
56 views

Dendrogram in hierarchical clustering

I have a confusion regarding the y-axis of the dendrogram. In many references, I see that the y-axis of the dendrogram is labeled as the Euclidean distance like the one shown below : But according to ...
  • 45
0 votes
0 answers
5 views

How to check how similar or different two texts are based on their cluster membership?

I have generated clusters based on crowdsourced ratings of a set of comments (collected from two different platforms). Each comment was rated by three different annotators using a likert scale of 3 ...
  • 1
0 votes
1 answer
16 views

Find clusters of industrial probe

I'm new to data science, but in these last months i have started to gather many samples of values coming from different industrial probes (temperature of water, pression, Kw consumed, etc). I have ...
0 votes
0 answers
30 views

Using mean absolute SHAP values as input into UMAP

Imagine you have a data set and you calculate SHAP values to understand feature importance. You then calculate the mean absolute SHAP values, as per this bar chart in the Interpretable ML book. If you ...
0 votes
0 answers
11 views

Why is exactly that k-mediods or PAM works well in a non-metric space

While brushing up on k-means and related topics I read about the K-mediod/PAM clustering algorithm. One of the key points why it was an interesting algorithm, compared to k-means was that it worked ...

1
2 3 4 5
28