Questions tagged [clustering]
Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.
1,092
questions
1
vote
1answer
5 views
Classification of multidimensional data to multidimensional clusters with a varying subcluster structure
I have a large dataset with mixed (numerical, categorical, textual) data that I need to classify. The clusters are well-defined, but multidimensional (i.e. vector-valued) and have a varying structure ...
0
votes
0answers
12 views
Three different errors using external information. Which one makes sense? (Or how to interpret each?)
I have a question on clustering and I think DSE is a good place for it. My goal is to compare clustering methods considering different method and different number of clusters using an external ...
0
votes
1answer
25 views
Clustering without information about identifier
I have a data-set with different products and binary value if it was sold in a store or not. I looks like:
...
3
votes
1answer
194 views
What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?
This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm?
I understand that EDA helps us generate good and helpful insights ...
-1
votes
1answer
11 views
How to group clusters with semantic similarity?
I have a list of job titles. I found the semantic similarity between them by using word2vec in spacy.
Now I want job titles ...
2
votes
1answer
16 views
How to cluster texts by most relevant words
I have a huge amount of documents and every document has its own portrait, where a portrait has this structure (document_id, word, weight). TFIDF, basically.
I want to cluster these documents into ...
0
votes
1answer
58 views
Scaling of ordinal data before both hierarchical and KMeans clustering
I am new to data analytics. As part of my assignment I have to perform both hierarchical and Kmeans clustering on a data set wherein all applicable variables are ordinal (1-5 rating scale). Do I need ...
1
vote
1answer
20 views
Clustering using both text and numerical features
I have a dataset that contains 2 types of features, one is generated from doc2vec and one is numerical feature. I would like to perform clustering analysis on them. However, due to the size of doc2vec ...
0
votes
1answer
329 views
Weighted clustering of lat lon coordinates
I have millions of lat long points that have been grouped into squares. Some squares have thousands of points, others have a couple of points. The idea is that we have one set of lat long for the ...
1
vote
0answers
15 views
What is the best way to cluster this kind of data?
I have data that looks like this:
The chart on the left is the trend and the smaller chart on the right is the box plot showing the distribution of means. Each color is the output of a particular ...
-1
votes
1answer
37 views
How do we visualize data in hierarchical clustering? [closed]
Can anybody tell me how to do visualization when applying hierarchical clustering to data with more than 2 features? Do we need to do dimensionality reduction before each clustering?
1
vote
1answer
32 views
DBSCAN on textual and numerical columns
I have a dataset which has two columns:
title price
sentence1 12
sentence2 13
I have used doc2vec to convert the ...
0
votes
1answer
30 views
Feature relevance in PCA + kmeans algorythm
Working on the World Happiness Report dataset, i have N countries with M features and a happiness score. This is the parameter I built 3 classes from: happy, medium, unhappy (numerical intervals of ...
0
votes
0answers
11 views
Discrete Wavelet Transform Time Series
My problem is to cluster some time series together. But due to a huge length I was interested in using some methods to reduce the dimensionality. I was thinking of Discrete Wavelet Transform since the ...
0
votes
1answer
9 views
Unsupervised Hierarchical Agglomerative Clustering
I've read a number of papers where the authors talk about "Unsupervised Hierarchical Agglomerative Clustering". They seem to imply that the algorithm determines the number of clusters based ...
0
votes
2answers
95 views
Topic models for non-textual data?
I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features.
For each observation, I combine the features into a single vector ...
0
votes
0answers
25 views
Considerations to take into account when clustering
My idea is to use clustering to perform stock segmentation based on risk, building different risk levels that might adapt better to different kind of users.
Hence I have computed different risk ...
0
votes
0answers
20 views
How to assign the remaining points to clusters [closed]
Very recently I was asked to find a way to assign new points on clusters formed with data in the past. I have two different ideas:
Compute the similarity of these data points
with the centers of ...
1
vote
0answers
9 views
How can the labels of AgglomerativeClustering be re-computed?
I'm using scikit learn's AgglomerativeClustering on a large data set.
I want to modify the distance_threshold after the model has already been computed. Computing ...
2
votes
0answers
14 views
Methods of de-emphasizing some dimensions in a cluster analysis
I'm trying to understand how "weightings" on different dimensions in a cluster analysis might relate to the range of values along a given dimension in the dataset.
DATA SET
List of 1,000 to ...
4
votes
1answer
193 views
Text classification based on n-grams and similarity
I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
3
votes
1answer
107 views
Finding dominating attributes with in the clusters generated
I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
1
vote
2answers
127 views
Notion of cluster centers and cluster comparison in Density Based Algorithms
I have done some research on clustering algorithms since for my goal is to cluster noisy data and identify outliers or small clusters as anomalies. I consider my data noisy because of my main ...
0
votes
0answers
31 views
Grouping data where each cluster has a similar sum of a variable?
I have a data set of postcode areas and their populations. I would like to cluster the data set in a way in which each cluster has roughly the same population of 80,000. Phrasing mathematically I'd ...
0
votes
1answer
16 views
What is the difference between spiral, flame, aggregation data
What is the difference between spiral, flame, aggregation data? What are the names of the columns, or what are the columns indicate?
For example, spiral is like to:
...
3
votes
2answers
242 views
Anomaly detection using clustering of highly correlated Categorical data
My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
2
votes
2answers
119 views
Techniques for Cluster Analysis of a Very Large (n=140000) Binary Dataset in Python?
In essence: what techniques in Python are possible to find clusters/trends in a very large categorical dataset?
My very large dataset (140000 rows/observations, 80 variables) of categorical data has ...
0
votes
1answer
23 views
Clusterize item set with items as vectors of features
I have to clusterize this dataset in which I have houses and water consumption in this form:
$$
House1 = (x_{1},x_{2}... x_{n});\\
House2 = (y_{1},y_{2}... y_{n});\\
House3 = (z_{1},z_{2}... z_{n});\\
...
0
votes
1answer
41 views
Any cluster algo can cluster time series datasets based on variation ratio(or quantity)?
I learn machine learning from sciki and read its documents.
Clustering cluster groups based on the euclidean distance and filter them by different ways ex: guassian distribution, or mean-shift...etc.
...
3
votes
2answers
527 views
Is k-means with Mahalanobis a valid option for clustering?
I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters.
The steps are:
Create aggregate datasets (...
33
votes
6answers
57k views
Calculating KL Divergence in Python
I am rather new to this and can't say I have a complete understanding of the theoretical concepts behind this. I am trying to calculate the KL Divergence between several lists of points in Python. I ...
3
votes
1answer
85 views
PCA and k-means for categorical variables?
I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
0
votes
1answer
54 views
Plotting clustered sentences in Python
I have the following three sentences, extracted from a dataframe. I would like to check the similarity and create clusters based on their level of similarity.
...
0
votes
1answer
62 views
clustering people according to answers on survey
Hi I am finding it hard to find online the best clustering algorithm for clustering people according to answers they gave on 20 question survey. There are four categories which each of these answers ...
-1
votes
1answer
12 views
Detect time pattern in sequence of events
I have a time series with a timestamp and an associated event:
Time
Event
1
A
2
B
3
C
T
A
I was wondering if there is a technique/method to figure out which events most often precede others in a ...
4
votes
2answers
12k views
How to classify and cluster this time series data [duplicate]
I have post already the question few months ago about my project that I'm starting to work on. This post can be see here:
Human activity recognition using smartphone data set problem
Now, I know ...
0
votes
0answers
6 views
Calculate Variance from Ward distances
The Ward method for clustering minimizes the total within-cluster variance.
So I suppose that there is a link between the Ward distances that I got with the linkage function, and the variance of the ...
0
votes
1answer
36 views
how to cluster inseparable data
Suppose I have a dataset containing two very similar classes of data. By similar, I mean that the 'distance' between these two classes is very small. For example, one instance in Class 1 is the sum of ...
4
votes
4answers
169 views
Algorithm for deriving multiple clusters
Suppose I have a set of data (with 2-dimensional feature space), and I want to obtain clusters from them. But I do not know how many clusters will be formed. Yet, I want separate clusters (The number ...
1
vote
1answer
21 views
Which data sets would help to predict (exponential) market trends?
Which kind of datasets do websites such as MeetGlimpse, trends.co, explodingtopics.com use to detect exponential market trends? I love them (not affiliated) and would like to better understand how ...
0
votes
0answers
16 views
Feature scaling for clustering
I want to cluster groups, using K-Means, DBSCAN, etc. algorithms, based on lat-lng coordinates along with other features such as dummy variables, continues variables (in different units).
What would ...
2
votes
2answers
30 views
Elasticsearch + Clustering
I'm currently working on a project that relies on the clustering of documents into an unknown number of clusters, based on a similarity threshold (ideally using cosine distance between tf-idf vectors)....
0
votes
1answer
147 views
How can i use Hellinger Distance on array of different length?
I have to use Hellinger distance to compare arrays that are not the same length.
How do you do this correctly? Putting a zero in the missing fields for the shorter array does not sound like the best ...
0
votes
0answers
7 views
Clustering of longitudinal user generated data; determine at what point in time does the user “become” the final clustering outcome
I analyse a lot of telephone call log data sets (akin to user generated data) and I use k-means clustering a lot to look at the types of callers that exist in the data set. The callers are clustered ...
-1
votes
1answer
36 views
Can clustering be applied on Linearly distributed data?
Let's say I have a sample dataset df generated as:
...
0
votes
1answer
39 views
How to increase number of outliers in a dataset?
I have a dataset with 1000 rows and 4 columns with 3 outliers .I want to add another 7 outliers related to them for detection by clustering.
...
-1
votes
1answer
22 views
Machine learning: inferring parents from child data
In producing SCADA systems, we get large lists of signals.
The signals can be a couple of thousands and all have verbal descriptions such as:
...
1
vote
1answer
42 views
How to use spectral clustering to predict?
In an academic paper, they talk about using a nearest neighbour algorithm to predict the cluster of a new point. And how the number of nearest neighbours is set to 10 in their example.
What do they ...
2
votes
1answer
30 views
How to retrain a K-Modes model based on daily data?
I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
1
vote
1answer
34 views
How to properly train your Self-Organized Map?
I stumbled recently upon the Self-Organized Map, an ANN architecture used to cluster high dimensional data, while simultaneously imposing a neighbourhood structure on it. It's trained through a ...
