Questions tagged [clustering]
Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.
1,394 questions
2
votes
1
answer
46
views
1D 'Clustering'
I’m working on a research paper about a robust federated learning aggregation scheme to defend against attackers.
I have a 1D array of trust scores for clients, and I need to cluster them to identify ...
- 21
1
vote
0
answers
14
views
How to Interpret Laplacian Scores for Feature Importance Ranking in Unsupervised Feature Clustering?
I am currently working on unsupervised feature importance ranking using graph clustering methods, specifically focusing on the Laplacian score as a metric. However, I am struggling to clarify the ...
- 11
0
votes
0
answers
11
views
Heuristics for hierarchical clustering with custom linkage function
I made my own custom linkage function for SciPy, and I want to add heuristics.
I cluster json sequences and for example if one cluster is sufficiently big (let’s say 20 Jsons) and the other one is ...
2
votes
1
answer
41
views
Detect Largest connected component in a scatter plot (on bivariate data)
I have bivariate data representing the position of an identified reference point along a y axis which ranges from -100 to 100 mm. When I plot this data in a scatter plot I can see outliers and groups ...
0
votes
0
answers
18
views
Is it possible to compute Davies Bouldin score from a precomputed distance matrix using sklearn?
I'm trying to compute the Davies Bouldin score to compare different clustering approach. I have a precomputed distance matrix (that represents edit-based distance between texts).
I'm using the scikit-...
- 1
0
votes
2
answers
96
views
Clique partition for an undirected weighted graph
I have an undirected graph and graph edges are weighted. I want to partition the graph into cliques. I don't know the number of cliques in advance.
These are the objectives:
Primary objective: The ...
0
votes
0
answers
34
views
How to Perform Clustering with OpenAI Embeddings Alongside Other Variables?
I’m working on a clustering project where my dataset includes both traditional variables (numerical and categorical features) and embeddings of multiple variables generated by OpenAI models. My goal ...
- 1
0
votes
0
answers
6
views
Deep cluster-specific feature selection?
As the title states, I was wondering if work exists on achieving feature selection based on the cluster the datapoint belongs to (e.g. each cluster has a different set of selected features).I know ...
- 1
-1
votes
1
answer
23
views
Applying the model to unseen data
What are good ways to extend an ML model with a new class without relabeling all previous data?
Problem with data representing classes that weren't present during supervised training
Suppose we ...
- 99
-1
votes
1
answer
50
views
Daily Pattern Clustering Methods for Time Series Data
I’m working on a project involving daily usage patterns of GN2 (Nitrogen Gas). My primary goal is to reduce the variance of daily patterns and minimize the gap between GN2 production and consumption ...
- 1
0
votes
0
answers
13
views
Agglomerative clustering with min and max cluster size constraints
Are there any python packages that have agglomerative clustering algorithms which have min and max cluster size constraints built in?
I found a great package called KMeansConstrained but unfortunately ...
- 113
0
votes
0
answers
11
views
I want to automate the process of putting the similar name files in a separate folder
I have a list of paths of all the folders in a subfolder , and some path_names have "Chapters"or"Chapter"or"chapter"or even "chaptser" , I want to detect these ...
0
votes
0
answers
10
views
Best methods to stratify data into 4 groups (unsupervised manner) using a set/combination of variables
I'm trying to stratify a set of patients according to possible molecular subtypes of cancer. Now, I know all these patients have a type of cancer, but the goal is to (in a unsupervised manner) cluster ...
- 101
1
vote
1
answer
42
views
What is this type of problem is this?
I have a set of entity types, say colors (red, green, blue, etc.) and a set of groups of entities. E.g. one group may be 3 blue, one may be 2 red and 1 blue, and so on. I have the assumption that ...
- 11
0
votes
0
answers
25
views
Finding clusters: DBSCAN or Friends-of-Friends?
In my undergraduate year, I search for dark matter halos in cosmological simulations using Friends-of-Friends algorithm.
https://swift.dur.ac.uk/docs/FriendsOfFriends/algorithm_description.html
Now I ...
- 101
0
votes
0
answers
17
views
Which approach suits for a problem where i try to find correspondance between two objectif of différent nature?
I struggle to qualify the problem i have and find suitable solution.
My problem is for exemple on one side i have invoices with amount due date invoices number and on thé other side i have transaction ...
- 101
0
votes
0
answers
16
views
Distance between a data point and a cluster
My dataset has 400 rows and 14 columns. I take rows 1-100 of data and find clusters using pca and t-SNE. Now, I want to see if any of rows 101-200 lie within any of the clusters I have found for rows ...
1
vote
0
answers
39
views
How to Determine the Optimal Number of Clusters / distance threshold in Agglomerative Clustering Using a Connectivity Matrix?
I am working on agglomerative clustering with the goal of ensuring continuity among clusters. To achieve this, I am using a connectivity matrix to enforce certain continuity constraints. However, it ...
- 11
0
votes
2
answers
22
views
Measure the significance of differences in internal distances in two clusters
I am a linguist studying grammatical variation in 169 female and 169 male Norwegian authors (using a treebank), based on 8 inflectional and syntactic properties. (Each property involves a choice ...
0
votes
0
answers
47
views
Identifying patterns and trends in insurance behavior with clustering or decision trees
I have a list of patient accounts based on their discharge date. I have various inputs related to each patient such as their financial class, insurance information, demographics, claims information, ...
0
votes
1
answer
24
views
Clustering method for frequency embeddings
I have, for example, the following lists of words that I want to cluster. The lists could have different lengths, and the vocabulary is $W = \{a,b,c\}$. The criteria of clustering 2 lists into a same ...
0
votes
0
answers
20
views
Spectral clustering with overlapping communities
Is there utility in replacing the Euclidean hard clustering heuristic (e.g., $k$-means) step in spectral clustering with a fuzzy clustering step to detect overlapping clusters? I am assuming this ...
- 1
0
votes
0
answers
15
views
Data Imputation and Cluster Analysis
Is there a preferred method of imputation for cluster analysis? Would the selection of imputation method be different if it were dbscan versus hierarchical clustering?
It seems there is an ongoing ...
- 133
0
votes
0
answers
4
views
Feature Group Weighting K-Means for Subspace clustering
I am working with fgkm algorithm in R shown in https://rdrr.io/cran/wskm/man/fgkm.html
I get exactly same result when grouped by different group. What is the possible explanation?
1
vote
0
answers
20
views
Geolocation Clustering - What is the right approach?
I am working on a research project on illegal dumping within a city & devicing advanced analytics solutions for the waste management company like resource optimisation, trend identification etc ...
0
votes
0
answers
18
views
Angular Distribution Function for Clustered Data Points
I was tasked recently with analyzing a cluster of individual points (each point representing a particulate), and I was advised that it would be good to apply an angular distribution function (ADF) on ...
1
vote
2
answers
752
views
DBScan for image segmentation and clustering: how does it work?
I think I have understood the DBScan algorithm for 2D data points. We can consider the example in scikit-learn. They generate a set of data points:
...
0
votes
1
answer
68
views
How to preprocess/encode categorical data, to use in dimensionality reduction and clustering algorithms?
I am working on a project witht the goal of clustering participants of in a survey according to their answers. The dataset is a set of 63 questions, some nominal and some ordinal. How should I encode ...
0
votes
0
answers
21
views
Understanding most important features from an additional column
I'm fairly new to data science in general and I'm doing some analysis. Let us say I have N rows and D features, and I have a ...
- 61
0
votes
0
answers
6
views
I have a confusion over the clustering, techniques involved and the scores. This is more about concept based since I am new to clustering models
What is astonishing to me is that the established norms for clustering data are not actually able to deduce the real results in my problem.
I created a K=2 clustering kmeans and kmeans-constrained (...
- 1
0
votes
0
answers
21
views
Calculating topic vector similarity based on document frequency
I am looking into ways to calculate the similarity of two topic vectors, where each dimension of a vector is a tuple made up a word describing the topic, along with the document frequency (tf-idf) of ...
- 91
0
votes
0
answers
12
views
Why the graph doesn't display the missing value even though values in data file are not missing?
i'm estimating a GARCH model , modeling the volatility in Bitcoin price and expected inflation, the graph is log diff of expected inflation proxy "market yield on treasury securities "
Thank ...
2
votes
1
answer
39
views
Grouping similar classes to improve accuracy, whilst maximising the number of classes
Suppose I have a large number of distinct classes, some of which are related.
My model has high classification accuracy for some classes, whilst other classes are hard to predict.
How could I group ...
- 816
0
votes
1
answer
131
views
Calculating weighted cosine similarity between vectors of words
I have two word lists, where each word is representative of each topic. A topic is created from a collection of documents (tweets in this case). Not all words would’ve appeared an equal number of ...
- 91
0
votes
0
answers
5
views
Leveraging Extra Data to Enhance Text Clustering
I want to cluster thousands of text data (called corpus A) and find a label for each cluster. Accuary of clustering is significantly important, because I want to use the texts and their labels for ...
1
vote
1
answer
67
views
Algorithm to cluster face vectors by person for an unknown number of people
I am building an open source framework for image processing. One of my demo programs takes a directory of photos, extracts the faces, clusters the face embeddings, and generates an HTML gallery ...
- 611
0
votes
0
answers
15
views
User Behaviour Anomaly Detection
I´m trying to detect anomalies in the behaviour of users in an app. My dataset have several fields, but I think the most important ones are User Id, TimeStamp, and Event_name.
So for example I could ...
1
vote
0
answers
142
views
Using UMAP on text data (euclidean distance on jaccard distance matrix)
I am checking the capabilities of the UMAP dimensionality reduction algorithm, I am not sure whether the approach I am using is valid and does not violate the rules/limitations of this algorithm.
...
- 21
0
votes
0
answers
25
views
How to organize multi-layer data in Orange Data Mining?
I have data in the form of a MATLAB cell array in which:
Rows are individual ROIs
columns are image channels
But each element of each column stores not only the mean intensity value of the ROI, but ...
1
vote
1
answer
53
views
Visualizing clusters of 3D images / volumes
I have a dataset of 3D (medical) images / volumes, think CT/MRI recordings. In other words, each of my samples consists of a 3D array of grayscale values, typically in the range of e.g. 1000x1000x500. ...
- 111
0
votes
0
answers
17
views
Need to compare results using Ward's method
So I create clusters like this and StandardScale them
...
- 1
0
votes
1
answer
27
views
Group/cluster semantically similar classes in reports?
I'm fine-tuning BERT models to binary classify reports. For example, a report can be about 'birds' or not about 'birds'.
This works really well, but now I want to do multi-label classification, ...
- 143
0
votes
0
answers
23
views
How to classify/recognize postage stamp varieties?
As a hobbiest stamp collector, I often run into the need for classifying stamps based on minute differences, such as these:
Now, I literally have thousands of them (in ziploc bags) and I am planning ...
0
votes
0
answers
19
views
Best value of K when using K-Nearest Neighbors with Spectral Clustering
I'm using scikit-learn's SpectralClustering class, which has the option of building its affinity matrix using a K-Nearest Neighbors algorithm. Is there any way to ...
0
votes
0
answers
19
views
Content based filtering with dissimilar user interests
I'm prototyping a system for recommending articles to users. A single embedding vector is generated for the article summary, and users self report a list of non-enumerated interests.
Concretely, a ...
0
votes
0
answers
55
views
Classifying Players as winners or losers
I have a dataset that I curated from a game that I play. There are currently 130 instances (i.e. players) and an innumerable number of features. Experience tells me <10 features would be sufficient....
- 35
0
votes
0
answers
14
views
Model for k means clustering of last mile logsitics data set
I want to utilize Orange for K means clustering for Network Design for Courier Company using K means clustering. Data set includes Longitude & latitude of delivery points, parcel weight, area type ...
- 1
0
votes
0
answers
7
views
How do I determine syndicate and collusion indication with clustering and network analysis on a large unlabeled user transaction data?
How do I determine syndicate and collusion indication with clustering and network analysis on a large unlabeled user transaction data?
So far, I've only been training with labeled data on fraud-...
0
votes
0
answers
13
views
Unsupervised Post-Clustering Feature Selection - Laplacian Score
I know that feature selection methods such as the Laplacian score or Fisher score are typically used for dimension reduction prior to clustering, but is there any reason why the same methods couldn't ...
- 203
2
votes
0
answers
16
views
Finding parameters which reveal clustering in t-SNE
These data are from SAMHSA, Mental Health Client-Level Data. I am trying to find the right parameters to obtain clustering as in this paper. Code here.
For now, I'm dropping columns which aren't ...
- 121