Questions tagged [clustering]
Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.
1,386
questions
0
votes
0
answers
8
views
How to solve classification problem that we should cluster elements, with Multinomial classification from CS229?
I just learned about Multinomial classification (CS229 Lecture note (What I learned is on page 24)) and I attempted to solve a problem that Obesity classification from Kaggle. Kaggle Link
I tried to ...
0
votes
0
answers
8
views
Efficiently cluster Gaussian vectors
Let $X^i,1\leq i\leq n$ iid Gaussian vectors of $\mathbb{R}^p$. I want to cluster the coordinates $1\leq j\leq p$ knowing the following facts:
Each coordinate $X^i_j$ is centred and has variance $1$, ...
- 101
0
votes
0
answers
5
views
Clustering high-dimensional data
Every sample in my dataset consists of two components: its $x$ component is a $n\times m$ matrix, and its $y$ component is a $p\times q$ matrix (Feature selection and PCA have been done before). Apart ...
- 1
0
votes
0
answers
8
views
Confusion about dendrogram
Consider the following dendrogram in aglomerative clustering :
The distance of merging the cluster containing the points p1 , p2 and p3 with the cluster containing the points p4, p5 and p6 is this ...
- 221
1
vote
0
answers
32
views
Looking for a way to train a model to learn parameters for clustering
I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension.
Now, I'm applying clustering to these points for each review. The success of my model depends ...
- 33
1
vote
1
answer
42
views
Beginner basic clustering model and one-hot encoding?
I have a dataframe of natural disaster incidents in Afghanistan from 2016 - 2023.
Column names:
REGION (Northern, Eastern etc)
PROV_CODE (province)
PROV_NAME
DIST_CODE (district)
DIST_NAME
INC_DATE (...
- 13
0
votes
1
answer
29
views
Beginner clustering project, what are the input features and how do I analyze the data?
I am a beginner to data science. I have this dataset on natural disaster events in Afghanistan from 2016 - 2017. Columns:
REGION (ex. North, North West, etc)
PROVINCE_NAME (kind of like US 50 states)
...
- 13
2
votes
2
answers
63
views
How to deal with high data volumes? (Tools, techniques, concepts, etc.)
I have some doubts about how to deal with high volumes of data. I'm currently working in the data analysis/data science field, so I've had the chance to perform calculations, manipulate data, and ...
- 21
0
votes
0
answers
12
views
Clustering of two datasets in different years
I want to analyze two datasets by running a clustering algorithm on both and comparing the results. The two datasets have the same variables. The only difference is that one dataset is from 2010 and ...
0
votes
1
answer
29
views
Outlier filtering from time series data
I have time series data that I eventually want to cluster after using dimensionality reduction. I am thinking about how to handle outliers. The data has seasonal/periodic patterns. I have tried IQR ...
- 1
0
votes
0
answers
19
views
Encoding soft clustering results as features
I want to use cluster numbers from soft clustering algorithm output as a some sort of categorical feature (or features), add them to other features for further training in another model (Y's from soft ...
- 13
0
votes
0
answers
4
views
Clustering or Finding Similarities Among Portfolio Allocations
I am trying to cluster a set of portfolios with percentage allocations among Stock, Bond, Other, and Cash. I am not sure what's the best way to go about this because the variables are interdependent ...
0
votes
0
answers
55
views
Identifying recurring transaction clusters (subscriptions) at a user level
I need some help converting this issue into a machine learning problem.
Goal:
Grouping credit charges into clusters of recurring transactions per user
Input data:
List of credit card charges with
<...
0
votes
0
answers
26
views
divide a large group of people into subgroups based on two parameter
note in advance: I'm new to data-analysis and although my major civil engineering taught me about statistics, I did not apply it the way I would have encountered in real life, or in this field of work....
0
votes
0
answers
34
views
Does clustering belong to the domain of data mining or to the domain of machine learning?
Question 1. Does clustering belong to the domain of data mining or to the domain of machine learning? Or to both domains?
Question 2. Depending on the answer to Question 1, could you please suggest a ...
- 101
0
votes
1
answer
60
views
Is this considered a good visualization of clustered data?
I have used sckit-learn to cluster data using K-means (3 clusters). I have an issue with considering the plot as "readable" since clusters are overlapping (even when encircling them)
Any ...
- 17
1
vote
2
answers
89
views
Cluster/Similarity problem with two datasets of different cardinality
I want to cluster financial products according to their similarity. I have two dataset of different cardinality:
One-to-One dataset: One ID has One attribute/feature per column - Describes a ...
- 550
0
votes
0
answers
19
views
Building an application to group/label browser tabs
I'm trying to develop an application where users can click a button and all of the open tabs in their browser will be placed into tab groupings based on similarity of the tab. Microsoft Edge has a ...
- 1
0
votes
0
answers
42
views
Using "precomputed" distance matrices as input to scikit-learn clustering metrics
Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
- 21
0
votes
0
answers
28
views
How do I group data to have a similar range of levels?
Background
I am working on a procurement bid where we are purchasing a number of different vehicle types for our various locations. As part of the analysis, I want to plot the prices of vehicle type ...
- 113
2
votes
1
answer
100
views
scaling before hierarchical clustering by single and complete linkage
I know that for hierarchical clustering, it's the best practice to scale before so that you give the same weight to each variable. Otherwise, for the complete linkage, the variable with a wider range ...
- 21
3
votes
1
answer
62
views
Solve tough clustering problem with overlapping clusters
I'm having some trouble to solve a hard clustering problem.
I have a 2D dataset characterized by non spherical and partially overlaping clusters with different densities.
I've read a lot about ...
0
votes
0
answers
8
views
A query with regard to removing data from a dataset before clustering. - conceptual
I am in posession of data with regard to my domain which is energy economics. The dataset contains daily data on daily electricity demand along with the daily capacities of wind and solar plants for ...
0
votes
1
answer
83
views
Is there a way to automatically split large clusters that are greater than some maximum number of points?
I ran HDBSCAN on these coordinates and got some clusters but some are too large. HDBSCAN has a minimum cluster size parameter, but no maximum size. All I want is to intuitively divide larger clusters ...
0
votes
0
answers
14
views
Clustering algorithm for rectangular region detection
I have a Cartesian grid (with integer coordinates) that contains some samples. The total number of samples on the grid is expected to be in the millions, but the grid can extend upwards of a million ...
- 203
0
votes
0
answers
11
views
clustering droplet data along trajectories
I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
0
votes
2
answers
42
views
Text Classification Taking too long
I have a sample of 135k documents that are preprocessed, and to which I calculated TFIDF. I tried clustering with KMeans, which gave me a memory problem (20GB). Then, i tried with MiniBatch K-Means ...
0
votes
0
answers
13
views
Best distance metric and estadarization method for clustering with percentages data
I'm studying access patterns to a facility with clustering.
My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
- 101
1
vote
0
answers
14
views
Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation
Question:
I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
- 11
0
votes
0
answers
7
views
What are some standard methods for studying co-ocurrence patterns?
We have a series of academic articles annotated with several tags eg "environmental issues", "legal issues", etc.
I wish to understand whether some of these topics are frequently ...
- 161
0
votes
0
answers
11
views
Examples of distance "hyperparameters" used in clustering
From what I've seen in clustering, distance is taken as a hyper parameter (which is to be selected) when inferring the relationships/clusters between points.
What are some examples of highly-cited ...
- 123
0
votes
0
answers
12
views
Does scikit learns implementation of silhouette score support parallelization and will benefit from multiple CPUs?
I wish to use the silhuette score to get the optimum number of clusters. I know kmeans implementation in scikit learn supports parallelization. But I am unsure whether the same is true for silhouette ...
0
votes
0
answers
11
views
Detect Recent Anomalies
I'm trying to see how to approach this problem: I have a dataset of fraud transactions. There are several categorical columns, like country, type, merchant, etc. All of the records are considered ...
- 131
0
votes
3
answers
161
views
Clusterise a group of answers taking the question as relevant information
I am solving a problem where I group answers to a given question into clusters using k-means algorithm. The steps I follow are:
For every answer I get the corresponding vector. Reduce the vector ...
- 51
0
votes
0
answers
38
views
Perform k-means clustering over multiple columns using python but without library sklearn.kmeans
I'm on a project with the topic "clustering" using the KMeans method.
so I have a data set with 10 columns and 1000 rows divided into 5 clusters (in this case there are 5 robot movements). I'...
0
votes
0
answers
6
views
How to cluster paths in order to discover working area shape?
I have a robot that moves in a 2D space and records its position in time as a list of (x,y) coordinates. The 2D space is divided in sub-areas that cannot be automatically crossed. When a given event ...
- 199
0
votes
1
answer
65
views
Standard metric for distance between two clusters
Let $A=\{A_1,A_2,\cdots,A_m\}$ and $B=\{B_1,B_2,\cdots,B_n\}$ be two sets of points in $k$-dimensional Euclidean space. Each points $A_i$ or $B_i$ can be thought of as a feature vector of a data ...
- 1
0
votes
0
answers
16
views
Good distance metric for clustering frequency variables
I have a dataset of customers, with variables refering to ABSOLUTE FREQUENCIES (counts). For example, I have the number of times he makes a purchase in weekend, the number of purchases in laborable ...
- 101
1
vote
0
answers
81
views
How to do search/cluster over a million points?
I've a practical question in the areas of clustering/semantic search and would like to get some thoughts. Refer the figure for more details on this hypothetical situation.
Imagine I've 2 query ...
0
votes
0
answers
26
views
What dimensional reduction and similarity score work for sentence embeddings created using sentence transformers
I am clustering sentence embeddings for log files, and find anomalies.
So, when I create sentence embeddings for logs using sentence transformers.
It will create vector of fixed length, which somehow ...
- 23
1
vote
1
answer
206
views
What is the most straightforward way to run python code in a chrome extension?
I am currently trying to develop a chrome extension for a project. My knowledge of Javascript is quite limited and this is my first using it, so I am trying to reduce as much as possible its use. The ...
- 11
0
votes
0
answers
62
views
How to choose the weights for a cluster analysis with gower's distances
I am doing a mixed data cluster analysis with 32 variables. For this reason i chose to use gower's distances for clustering. The variables can be divided into three groups with different sizes. The ...
- 1
1
vote
2
answers
83
views
Hierarchical Clustering: Dendograms
What are dendograms? How de we interpret them? By looking at the dendograms, how can we decide the number of clusters to be formed?
- 11
0
votes
0
answers
13
views
A visually pleasing way of displaying the cluster assignment of a sample with respect to time
What are some of the more visually appealing ways to represent cluster assignments of a sample?
Usually the scatter plot works well but in this instace I am more insterested as to which cluster the ...
- 1
0
votes
2
answers
73
views
Clustering with a "catchall" cluster
I have a dataset which consists of 2 types of items. One type (Type A) is easily matchable, and there are several groups within this set I would like to cluster. The second type (Type B) is randomly ...
1
vote
0
answers
46
views
How does HDBSCAN generate cluster hierarchy?
HDBSCAN generates the minimum spanning tree where each vertex represents the data point whereas the edges represent the mutual reachability distance.
But how does it generate the cluster hierarchy?
I ...
0
votes
1
answer
25
views
Clustering and grouping noisy time series fuel data
I have a time series of fuel data which are split into distinct clusters (as shown in the image below. I'm looking at the green line). The clusters almost always have a descending gradient. I would ...
0
votes
1
answer
81
views
Clustering task: drop or not drop a categorical attribute/feature for which each row in the dataset contains a different value
I am dealing with a clustering task. In the dataset I am using there is a categorical feature and for each row in the dataset I have a different value for that feature (my dataset consists of 1000 ...
- 1
0
votes
1
answer
50
views
Clustering: is it common that data just cannot be grouped?
I'm currently in the middle of a clustering project and struggling to get acceptable results, is it commonplace that datasets just can't be clustered?
Context:
I'm trying to cluster a relatively small ...
- 101
0
votes
0
answers
7
views
Expectation Step in Gaussian Mixture Model for Matrix Data Not Producing Proper Posterior Probabilities
I'm working on implementing a Gaussian Mixture Model (GMM) for three-way data (i.e., a set of matrices) in R. The GMM is being estimated using the Expectation-Maximization (EM) algorithm. However, I'm ...
