Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
1
vote
1answer
5 views

Classification of multidimensional data to multidimensional clusters with a varying subcluster structure

I have a large dataset with mixed (numerical, categorical, textual) data that I need to classify. The clusters are well-defined, but multidimensional (i.e. vector-valued) and have a varying structure ...
0
votes
0answers
12 views

Three different errors using external information. Which one makes sense? (Or how to interpret each?)

I have a question on clustering and I think DSE is a good place for it. My goal is to compare clustering methods considering different method and different number of clusters using an external ...
0
votes
1answer
25 views

Clustering without information about identifier

I have a data-set with different products and binary value if it was sold in a store or not. I looks like: ...
3
votes
1answer
194 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
-1
votes
1answer
11 views

How to group clusters with semantic similarity?

I have a list of job titles. I found the semantic similarity between them by using word2vec in spacy. Now I want job titles ...
2
votes
1answer
16 views

How to cluster texts by most relevant words

I have a huge amount of documents and every document has its own portrait, where a portrait has this structure (document_id, word, weight). TFIDF, basically. I want to cluster these documents into ...
0
votes
1answer
58 views

Scaling of ordinal data before both hierarchical and KMeans clustering

I am new to data analytics. As part of my assignment I have to perform both hierarchical and Kmeans clustering on a data set wherein all applicable variables are ordinal (1-5 rating scale). Do I need ...
1
vote
1answer
20 views

Clustering using both text and numerical features

I have a dataset that contains 2 types of features, one is generated from doc2vec and one is numerical feature. I would like to perform clustering analysis on them. However, due to the size of doc2vec ...
0
votes
1answer
329 views

Weighted clustering of lat lon coordinates

I have millions of lat long points that have been grouped into squares. Some squares have thousands of points, others have a couple of points. The idea is that we have one set of lat long for the ...
1
vote
0answers
15 views

What is the best way to cluster this kind of data?

I have data that looks like this: The chart on the left is the trend and the smaller chart on the right is the box plot showing the distribution of means. Each color is the output of a particular ...
-1
votes
1answer
37 views

How do we visualize data in hierarchical clustering? [closed]

Can anybody tell me how to do visualization when applying hierarchical clustering to data with more than 2 features? Do we need to do dimensionality reduction before each clustering?
1
vote
1answer
32 views

DBSCAN on textual and numerical columns

I have a dataset which has two columns: title price sentence1 12 sentence2 13 I have used doc2vec to convert the ...
0
votes
1answer
30 views

Feature relevance in PCA + kmeans algorythm

Working on the World Happiness Report dataset, i have N countries with M features and a happiness score. This is the parameter I built 3 classes from: happy, medium, unhappy (numerical intervals of ...
0
votes
0answers
11 views

Discrete Wavelet Transform Time Series

My problem is to cluster some time series together. But due to a huge length I was interested in using some methods to reduce the dimensionality. I was thinking of Discrete Wavelet Transform since the ...
0
votes
1answer
9 views

Unsupervised Hierarchical Agglomerative Clustering

I've read a number of papers where the authors talk about "Unsupervised Hierarchical Agglomerative Clustering". They seem to imply that the algorithm determines the number of clusters based ...
0
votes
2answers
95 views

Topic models for non-textual data?

I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features. For each observation, I combine the features into a single vector ...
0
votes
0answers
25 views

Considerations to take into account when clustering

My idea is to use clustering to perform stock segmentation based on risk, building different risk levels that might adapt better to different kind of users. Hence I have computed different risk ...
0
votes
0answers
20 views

How to assign the remaining points to clusters [closed]

Very recently I was asked to find a way to assign new points on clusters formed with data in the past. I have two different ideas: Compute the similarity of these data points with the centers of ...
1
vote
0answers
9 views

How can the labels of AgglomerativeClustering be re-computed?

I'm using scikit learn's AgglomerativeClustering on a large data set. I want to modify the distance_threshold after the model has already been computed. Computing ...
2
votes
0answers
14 views

Methods of de-emphasizing some dimensions in a cluster analysis

I'm trying to understand how "weightings" on different dimensions in a cluster analysis might relate to the range of values along a given dimension in the dataset. DATA SET List of 1,000 to ...
4
votes
1answer
193 views

Text classification based on n-grams and similarity

I have tried to cluster hundred texts using k-means clustering. I would like to consider other algorithms to group text based on their content and try to spot news not related to other news (topic ...
3
votes
1answer
107 views

Finding dominating attributes with in the clusters generated

I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
1
vote
2answers
127 views

Notion of cluster centers and cluster comparison in Density Based Algorithms

I have done some research on clustering algorithms since for my goal is to cluster noisy data and identify outliers or small clusters as anomalies. I consider my data noisy because of my main ...
0
votes
0answers
31 views

Grouping data where each cluster has a similar sum of a variable?

I have a data set of postcode areas and their populations. I would like to cluster the data set in a way in which each cluster has roughly the same population of 80,000. Phrasing mathematically I'd ...
0
votes
1answer
16 views

What is the difference between spiral, flame, aggregation data

What is the difference between spiral, flame, aggregation data? What are the names of the columns, or what are the columns indicate? For example, spiral is like to: ...
3
votes
2answers
242 views

Anomaly detection using clustering of highly correlated Categorical data

My data has two columns and both are highly correlated e.g. if column1 has value ABC, column2 should be XYZ i.e. ABC-->XYZ. If column2 has anything else its Anomaly. Likewise there are thousands of ...
2
votes
2answers
119 views

Techniques for Cluster Analysis of a Very Large (n=140000) Binary Dataset in Python?

In essence: what techniques in Python are possible to find clusters/trends in a very large categorical dataset? My very large dataset (140000 rows/observations, 80 variables) of categorical data has ...
0
votes
1answer
23 views

Clusterize item set with items as vectors of features

I have to clusterize this dataset in which I have houses and water consumption in this form: $$ House1 = (x_{1},x_{2}... x_{n});\\ House2 = (y_{1},y_{2}... y_{n});\\ House3 = (z_{1},z_{2}... z_{n});\\ ...
0
votes
1answer
41 views

Any cluster algo can cluster time series datasets based on variation ratio(or quantity)?

I learn machine learning from sciki and read its documents. Clustering cluster groups based on the euclidean distance and filter them by different ways ex: guassian distribution, or mean-shift...etc. ...
3
votes
2answers
527 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
33
votes
6answers
57k views

Calculating KL Divergence in Python

I am rather new to this and can't say I have a complete understanding of the theoretical concepts behind this. I am trying to calculate the KL Divergence between several lists of points in Python. I ...
3
votes
1answer
85 views

PCA and k-means for categorical variables?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
0
votes
1answer
54 views

Plotting clustered sentences in Python

I have the following three sentences, extracted from a dataframe. I would like to check the similarity and create clusters based on their level of similarity. ...
0
votes
1answer
62 views

clustering people according to answers on survey

Hi I am finding it hard to find online the best clustering algorithm for clustering people according to answers they gave on 20 question survey. There are four categories which each of these answers ...
-1
votes
1answer
12 views

Detect time pattern in sequence of events

I have a time series with a timestamp and an associated event: Time Event 1 A 2 B 3 C T A I was wondering if there is a technique/method to figure out which events most often precede others in a ...
4
votes
2answers
12k views

How to classify and cluster this time series data [duplicate]

I have post already the question few months ago about my project that I'm starting to work on. This post can be see here: Human activity recognition using smartphone data set problem Now, I know ...
0
votes
0answers
6 views

Calculate Variance from Ward distances

The Ward method for clustering minimizes the total within-cluster variance. So I suppose that there is a link between the Ward distances that I got with the linkage function, and the variance of the ...
0
votes
1answer
36 views

how to cluster inseparable data

Suppose I have a dataset containing two very similar classes of data. By similar, I mean that the 'distance' between these two classes is very small. For example, one instance in Class 1 is the sum of ...
4
votes
4answers
169 views

Algorithm for deriving multiple clusters

Suppose I have a set of data (with 2-dimensional feature space), and I want to obtain clusters from them. But I do not know how many clusters will be formed. Yet, I want separate clusters (The number ...
1
vote
1answer
21 views

Which data sets would help to predict (exponential) market trends?

Which kind of datasets do websites such as MeetGlimpse, trends.co, explodingtopics.com use to detect exponential market trends? I love them (not affiliated) and would like to better understand how ...
0
votes
0answers
16 views

Feature scaling for clustering

I want to cluster groups, using K-Means, DBSCAN, etc. algorithms, based on lat-lng coordinates along with other features such as dummy variables, continues variables (in different units). What would ...
2
votes
2answers
30 views

Elasticsearch + Clustering

I'm currently working on a project that relies on the clustering of documents into an unknown number of clusters, based on a similarity threshold (ideally using cosine distance between tf-idf vectors)....
0
votes
1answer
147 views

How can i use Hellinger Distance on array of different length?

I have to use Hellinger distance to compare arrays that are not the same length. How do you do this correctly? Putting a zero in the missing fields for the shorter array does not sound like the best ...
0
votes
0answers
7 views

Clustering of longitudinal user generated data; determine at what point in time does the user “become” the final clustering outcome

I analyse a lot of telephone call log data sets (akin to user generated data) and I use k-means clustering a lot to look at the types of callers that exist in the data set. The callers are clustered ...
-1
votes
1answer
36 views

Can clustering be applied on Linearly distributed data?

Let's say I have a sample dataset df generated as: ...
0
votes
1answer
39 views

How to increase number of outliers in a dataset?

I have a dataset with 1000 rows and 4 columns with 3 outliers .I want to add another 7 outliers related to them for detection by clustering. ...
-1
votes
1answer
22 views

Machine learning: inferring parents from child data

In producing SCADA systems, we get large lists of signals. The signals can be a couple of thousands and all have verbal descriptions such as: ...
1
vote
1answer
42 views

How to use spectral clustering to predict?

In an academic paper, they talk about using a nearest neighbour algorithm to predict the cluster of a new point. And how the number of nearest neighbours is set to 10 in their example. What do they ...
2
votes
1answer
30 views

How to retrain a K-Modes model based on daily data?

I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
1
vote
1answer
34 views

How to properly train your Self-Organized Map?

I stumbled recently upon the Self-Organized Map, an ANN architecture used to cluster high dimensional data, while simultaneously imposing a neighbourhood structure on it. It's trained through a ...

1
2 3 4 5
22