Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

How to solve classification problem that we should cluster elements, with Multinomial classification from CS229?

I just learned about Multinomial classification (CS229 Lecture note (What I learned is on page 24)) and I attempted to solve a problem that Obesity classification from Kaggle. Kaggle Link I tried to ...
Gosu Choi's user avatar
0 votes
0 answers
8 views

Efficiently cluster Gaussian vectors

Let $X^i,1\leq i\leq n$ iid Gaussian vectors of $\mathbb{R}^p$. I want to cluster the coordinates $1\leq j\leq p$ knowing the following facts: Each coordinate $X^i_j$ is centred and has variance $1$, ...
kaleidoscop's user avatar
0 votes
0 answers
5 views

Clustering high-dimensional data

Every sample in my dataset consists of two components: its $x$ component is a $n\times m$ matrix, and its $y$ component is a $p\times q$ matrix (Feature selection and PCA have been done before). Apart ...
yuxuan-z's user avatar
0 votes
0 answers
8 views

Confusion about dendrogram

Consider the following dendrogram in aglomerative clustering : The distance of merging the cluster containing the points p1 , p2 and p3 with the cluster containing the points p4, p5 and p6 is this ...
John adams's user avatar
1 vote
0 answers
32 views

Looking for a way to train a model to learn parameters for clustering

I have 5000 docs, each is a review. For each review, i'm plotting the sentences in a semantic dimension. Now, I'm applying clustering to these points for each review. The success of my model depends ...
Prithvi's user avatar
  • 33
1 vote
1 answer
42 views

Beginner basic clustering model and one-hot encoding?

I have a dataframe of natural disaster incidents in Afghanistan from 2016 - 2023. Column names: REGION (Northern, Eastern etc) PROV_CODE (province) PROV_NAME DIST_CODE (district) DIST_NAME INC_DATE (...
Mas's user avatar
  • 13
0 votes
1 answer
29 views

Beginner clustering project, what are the input features and how do I analyze the data?

I am a beginner to data science. I have this dataset on natural disaster events in Afghanistan from 2016 - 2017. Columns: REGION (ex. North, North West, etc) PROVINCE_NAME (kind of like US 50 states) ...
Mas's user avatar
  • 13
2 votes
2 answers
63 views

How to deal with high data volumes? (Tools, techniques, concepts, etc.)

I have some doubts about how to deal with high volumes of data. I'm currently working in the data analysis/data science field, so I've had the chance to perform calculations, manipulate data, and ...
tms's user avatar
  • 21
0 votes
0 answers
12 views

Clustering of two datasets in different years

I want to analyze two datasets by running a clustering algorithm on both and comparing the results. The two datasets have the same variables. The only difference is that one dataset is from 2010 and ...
Ahmad Bhatti's user avatar
0 votes
1 answer
29 views

Outlier filtering from time series data

I have time series data that I eventually want to cluster after using dimensionality reduction. I am thinking about how to handle outliers. The data has seasonal/periodic patterns. I have tried IQR ...
Jim A's user avatar
  • 1
0 votes
0 answers
19 views

Encoding soft clustering results as features

I want to use cluster numbers from soft clustering algorithm output as a some sort of categorical feature (or features), add them to other features for further training in another model (Y's from soft ...
franz-german's user avatar
0 votes
0 answers
4 views

Clustering or Finding Similarities Among Portfolio Allocations

I am trying to cluster a set of portfolios with percentage allocations among Stock, Bond, Other, and Cash. I am not sure what's the best way to go about this because the variables are interdependent ...
Ahmad Bhatti's user avatar
0 votes
0 answers
55 views

Identifying recurring transaction clusters (subscriptions) at a user level

I need some help converting this issue into a machine learning problem. Goal: Grouping credit charges into clusters of recurring transactions per user Input data: List of credit card charges with <...
Fintech Pikachu's user avatar
0 votes
0 answers
26 views

divide a large group of people into subgroups based on two parameter

note in advance: I'm new to data-analysis and although my major civil engineering taught me about statistics, I did not apply it the way I would have encountered in real life, or in this field of work....
Abdulrahman Sheikho's user avatar
0 votes
0 answers
34 views

Does clustering belong to the domain of data mining or to the domain of machine learning?

Question 1. Does clustering belong to the domain of data mining or to the domain of machine learning? Or to both domains? Question 2. Depending on the answer to Question 1, could you please suggest a ...
Ommo's user avatar
  • 101
0 votes
1 answer
60 views

Is this considered a good visualization of clustered data?

I have used sckit-learn to cluster data using K-means (3 clusters). I have an issue with considering the plot as "readable" since clusters are overlapping (even when encircling them) Any ...
SSSOF's user avatar
  • 17
1 vote
2 answers
89 views

Cluster/Similarity problem with two datasets of different cardinality

I want to cluster financial products according to their similarity. I have two dataset of different cardinality: One-to-One dataset: One ID has One attribute/feature per column - Describes a ...
Maeaex1's user avatar
  • 550
0 votes
0 answers
19 views

Building an application to group/label browser tabs

I'm trying to develop an application where users can click a button and all of the open tabs in their browser will be placed into tab groupings based on similarity of the tab. Microsoft Edge has a ...
Ben's user avatar
  • 1
0 votes
0 answers
42 views

Using "precomputed" distance matrices as input to scikit-learn clustering metrics

Is there any validity to using a distance matrix instead of the raw points with metrics such as davies_bouldin_score and ...
Chris Coffee's user avatar
0 votes
0 answers
28 views

How do I group data to have a similar range of levels?

Background I am working on a procurement bid where we are purchasing a number of different vehicle types for our various locations. As part of the analysis, I want to plot the prices of vehicle type ...
ScottyJ's user avatar
  • 113
2 votes
1 answer
100 views

scaling before hierarchical clustering by single and complete linkage

I know that for hierarchical clustering, it's the best practice to scale before so that you give the same weight to each variable. Otherwise, for the complete linkage, the variable with a wider range ...
user154385's user avatar
3 votes
1 answer
62 views

Solve tough clustering problem with overlapping clusters

I'm having some trouble to solve a hard clustering problem. I have a 2D dataset characterized by non spherical and partially overlaping clusters with different densities. I've read a lot about ...
Lorenço Santos's user avatar
0 votes
0 answers
8 views

A query with regard to removing data from a dataset before clustering. - conceptual

I am in posession of data with regard to my domain which is energy economics. The dataset contains daily data on daily electricity demand along with the daily capacities of wind and solar plants for ...
user154329's user avatar
0 votes
1 answer
83 views

Is there a way to automatically split large clusters that are greater than some maximum number of points?

I ran HDBSCAN on these coordinates and got some clusters but some are too large. HDBSCAN has a minimum cluster size parameter, but no maximum size. All I want is to intuitively divide larger clusters ...
Ben Hendel's user avatar
0 votes
0 answers
14 views

Clustering algorithm for rectangular region detection

I have a Cartesian grid (with integer coordinates) that contains some samples. The total number of samples on the grid is expected to be in the millions, but the grid can extend upwards of a million ...
Mate de Vita's user avatar
0 votes
0 answers
11 views

clustering droplet data along trajectories

I have regular t,x,y,z-datapoints of droplets, that move continuously from top to bottom on different but similar paths. The droplets also have a volume that is approximately measured, they can appear ...
Jonas Hilti's user avatar
0 votes
2 answers
42 views

Text Classification Taking too long

I have a sample of 135k documents that are preprocessed, and to which I calculated TFIDF. I tried clustering with KMeans, which gave me a memory problem (20GB). Then, i tried with MiniBatch K-Means ...
ayowhatthedogdoin's user avatar
0 votes
0 answers
13 views

Best distance metric and estadarization method for clustering with percentages data

I'm studying access patterns to a facility with clustering. My variables are percentages. For example, for each user, I have the percentage of access 'in time' versus late, or the percentage of using ...
Kaikus's user avatar
  • 101
1 vote
0 answers
14 views

Clustering Similar Articles Using Mixed Data: Seeking Advice and Validation

Question: I'm working on a project where I need to cluster a dataset of articles based on various features, including text, numeric values, and categorical data. I've implemented a clustering approach ...
sara sara's user avatar
0 votes
0 answers
7 views

What are some standard methods for studying co-ocurrence patterns?

We have a series of academic articles annotated with several tags eg "environmental issues", "legal issues", etc. I wish to understand whether some of these topics are frequently ...
Jsevillamol's user avatar
0 votes
0 answers
11 views

Examples of distance "hyperparameters" used in clustering

From what I've seen in clustering, distance is taken as a hyper parameter (which is to be selected) when inferring the relationships/clusters between points. What are some examples of highly-cited ...
ABIM's user avatar
  • 123
0 votes
0 answers
12 views

Does scikit learns implementation of silhouette score support parallelization and will benefit from multiple CPUs?

I wish to use the silhuette score to get the optimum number of clusters. I know kmeans implementation in scikit learn supports parallelization. But I am unsure whether the same is true for silhouette ...
Ali Raheel's user avatar
0 votes
0 answers
11 views

Detect Recent Anomalies

I'm trying to see how to approach this problem: I have a dataset of fraud transactions. There are several categorical columns, like country, type, merchant, etc. All of the records are considered ...
F_M's user avatar
  • 131
0 votes
3 answers
161 views

Clusterise a group of answers taking the question as relevant information

I am solving a problem where I group answers to a given question into clusters using k-means algorithm. The steps I follow are: For every answer I get the corresponding vector. Reduce the vector ...
jesantana's user avatar
0 votes
0 answers
38 views

Perform k-means clustering over multiple columns using python but without library sklearn.kmeans

I'm on a project with the topic "clustering" using the KMeans method. so I have a data set with 10 columns and 1000 rows divided into 5 clusters (in this case there are 5 robot movements). I'...
Higashida Cyza's user avatar
0 votes
0 answers
6 views

How to cluster paths in order to discover working area shape?

I have a robot that moves in a 2D space and records its position in time as a list of (x,y) coordinates. The 2D space is divided in sub-areas that cannot be automatically crossed. When a given event ...
firion's user avatar
  • 199
0 votes
1 answer
65 views

Standard metric for distance between two clusters

Let $A=\{A_1,A_2,\cdots,A_m\}$ and $B=\{B_1,B_2,\cdots,B_n\}$ be two sets of points in $k$-dimensional Euclidean space. Each points $A_i$ or $B_i$ can be thought of as a feature vector of a data ...
govindah's user avatar
0 votes
0 answers
16 views

Good distance metric for clustering frequency variables

I have a dataset of customers, with variables refering to ABSOLUTE FREQUENCIES (counts). For example, I have the number of times he makes a purchase in weekend, the number of purchases in laborable ...
Kaikus's user avatar
  • 101
1 vote
0 answers
81 views

How to do search/cluster over a million points?

I've a practical question in the areas of clustering/semantic search and would like to get some thoughts. Refer the figure for more details on this hypothetical situation. Imagine I've 2 query ...
Namburi Srinath's user avatar
0 votes
0 answers
26 views

What dimensional reduction and similarity score work for sentence embeddings created using sentence transformers

I am clustering sentence embeddings for log files, and find anomalies. So, when I create sentence embeddings for logs using sentence transformers. It will create vector of fixed length, which somehow ...
Glinty's user avatar
  • 23
1 vote
1 answer
206 views

What is the most straightforward way to run python code in a chrome extension?

I am currently trying to develop a chrome extension for a project. My knowledge of Javascript is quite limited and this is my first using it, so I am trying to reduce as much as possible its use. The ...
Manuel's user avatar
  • 11
0 votes
0 answers
62 views

How to choose the weights for a cluster analysis with gower's distances

I am doing a mixed data cluster analysis with 32 variables. For this reason i chose to use gower's distances for clustering. The variables can be divided into three groups with different sizes. The ...
Fares's user avatar
  • 1
1 vote
2 answers
83 views

Hierarchical Clustering: Dendograms

What are dendograms? How de we interpret them? By looking at the dendograms, how can we decide the number of clusters to be formed?
int_x's user avatar
  • 11
0 votes
0 answers
13 views

A visually pleasing way of displaying the cluster assignment of a sample with respect to time

What are some of the more visually appealing ways to represent cluster assignments of a sample? Usually the scatter plot works well but in this instace I am more insterested as to which cluster the ...
Tam's user avatar
  • 1
0 votes
2 answers
73 views

Clustering with a "catchall" cluster

I have a dataset which consists of 2 types of items. One type (Type A) is easily matchable, and there are several groups within this set I would like to cluster. The second type (Type B) is randomly ...
Martin Thompson's user avatar
1 vote
0 answers
46 views

How does HDBSCAN generate cluster hierarchy?

HDBSCAN generates the minimum spanning tree where each vertex represents the data point whereas the edges represent the mutual reachability distance. But how does it generate the cluster hierarchy? I ...
Sushil Khadka's user avatar
0 votes
1 answer
25 views

Clustering and grouping noisy time series fuel data

I have a time series of fuel data which are split into distinct clusters (as shown in the image below. I'm looking at the green line). The clusters almost always have a descending gradient. I would ...
NewScreen20's user avatar
0 votes
1 answer
81 views

Clustering task: drop or not drop a categorical attribute/feature for which each row in the dataset contains a different value

I am dealing with a clustering task. In the dataset I am using there is a categorical feature and for each row in the dataset I have a different value for that feature (my dataset consists of 1000 ...
vito97's user avatar
  • 1
0 votes
1 answer
50 views

Clustering: is it common that data just cannot be grouped?

I'm currently in the middle of a clustering project and struggling to get acceptable results, is it commonplace that datasets just can't be clustered? Context: I'm trying to cluster a relatively small ...
roastbeeef's user avatar
0 votes
0 answers
7 views

Expectation Step in Gaussian Mixture Model for Matrix Data Not Producing Proper Posterior Probabilities

I'm working on implementing a Gaussian Mixture Model (GMM) for three-way data (i.e., a set of matrices) in R. The GMM is being estimated using the Expectation-Maximization (EM) algorithm. However, I'm ...
John Smith's user avatar

1
2 3 4 5
28