Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

2
votes
1answer
18 views

How to cluster/identify points away from a regression line

For many vine plots, I have NDVI and Leaf Area values for each vine. I already know that NDVI and LA has a strong positive correlation as you can see in this picture. But as you can see too, there ...
0
votes
2answers
25 views

Why does changing the cluster number change the plot in Kmeans?

This might be a dumb questions but I can't find the answer to it. I don't have the perfect mathematical understanding of kmeans, so apologies if it is. I'm just wondering why I see a different plot ...
0
votes
0answers
16 views

Transformation of matrix with missing values for hierarchical clustering

Comparing different variables, I got a matrix with lots of missing values. How do I have to transform the matrix below for hierarchical clustering? What I have already tried: ...
0
votes
2answers
26 views

Url string processing: what is the best way?

I have ~1000 different news websites and I scraped and saved all the internal url links for each website. For instance, the website dcgazette.com has a 2MB text file with associated urls: 1) https://...
-3
votes
0answers
22 views

Looking for a way to represent subsets with a vector of ternary values

I have a problem for which I believe there must be a simple machine learning solution, but I do not know how to search for it, so I am hoping that someone will be able to recommend an appropriate ...
-1
votes
2answers
18 views

Clustering of User Behavior in chatbot [closed]

I have a dataset that represents a user's behaviour in my chatbot (userid, last clicked button, session, timestamp). I need to divide the users into clusters based on their behaviour on the chatbot ...
0
votes
0answers
15 views

preprocessing : Predicting with Multiple+Multivariate+Multitrend time series data

I am trying to predict the value of a variable in a multivariate time series; of which I have multiple time datasets (one system = one dataset containing 10 variables in time and average 120,000 rows) ...
0
votes
0answers
16 views

Feature selection or Dimension reduction in unsupervised learning

I'm trying to do Embedded clustering using kmeans. This is customer data, so it involves a lot of sentences, so I'm using the universal sentence encoder before clustering. But I should be doing a ...
-1
votes
1answer
17 views

Clustering not working as expected

I have clusters as shown in the picture below. The data is 2d : the two parameters are error and time. I tried using the following clustering algorithms: 1) kmeans:clusters are spherical. This algo ...
0
votes
1answer
32 views

Stationary time series for clustering algorithms

I have a set of time series data that I would like to feed into a clustering algorithm (like k-means, using dynamic time warping as the distance function). After standardizing the data with mean 0 and ...
-1
votes
2answers
29 views

Anomaly detection k-means in Time Series

I'm trying to use k-means to detect anomalies in the Amount column. I have the following part of my dataset: ...
0
votes
1answer
23 views

Clustering with custom criterion (minimum cluster weight)

Edit: following comment from @anony-mousse, I'm changing the question to search for a general clustering approach that matches this criterion (minimum weight per cluster). I am to use a clustering ...
1
vote
1answer
26 views

Clustering a set of vectors

Provided a set ($m$ no. of) of n-dimensional vectors what would be the correct unsupervised approach to cluster them? The vectors essentially represent patterns. For example: Set of vector is ...
1
vote
1answer
15 views

Which approach to select category based on keywords

I want to assign a certain category to a group of keywords. So i.e. people can upload images or videos, when they do this they can set keywords for this. These keywords are free to type so words can ...
1
vote
2answers
16 views

How to find vertical clusters in 1-D data

I have residuals of a multivariate time series data obtained from sensors on a server.spikes in the plots of residuals indicate abnormal server state. I want to cluster the data into vertical clusters ...
0
votes
1answer
14 views

How to identify new clusters that the training data has never seen

I have to identify the different operational states of a server. I have readings related to the different sensors of the server ( like temp sensor,fan speed sensor,job load sensor etc).The data I have ...
1
vote
2answers
20 views

Standardization After PCA for Kmean clustering

I want to apply Kmean for clustering after PCA dimensionality reduction. I have standardized data with StandardScaler before the PCA, then I want to train Kmeans for finding clusters. However, the ...
0
votes
1answer
24 views

How to reach time points clustering?

What I met a problem is I do time-series clustering, and I found the clustering result isn't ideal. I can't use elbow method to know what clustering result is good, that means I have no ways to watch ...
2
votes
1answer
32 views

How to scale or standardize data that is mostly 0 (ranges from 0-1)?

I am relatively new to data science and big data munging in general. I currently have various columns of data that range from $0-1$, but most of the values in each ...
3
votes
1answer
26 views

How can I get the diameter of each community

I am trying to calculate the diameter of each community in my dataset, Zachary's karate club using Jupyter. I created a loop to iterate through, but it gives me the diameter of the whole network ...
2
votes
1answer
24 views

how to balance the data set with different number of observations

I am pretty new to ds field and recently I was working on a self project of the clustering model. My goal is to create clusters and see in each cluster what the common features are between customers. ...
0
votes
1answer
21 views

How to tune parameters batch by batch?

As the title states, I am trying to cluster a huge dataset and cluster it by using sklearn.Birch to learn incrementally. If it's a small dataset, I could just use ...
1
vote
2answers
25 views

Clustering on imbalanced data that has high correlation

I am clustering images of two categories, but for the purposes of the experiment, I do not know the labels i.e. this is an unsupervised problem. Via ...
0
votes
0answers
10 views

Algorithm/approach to predict multiple questions in a survey

I have I set of 6k surveys about personal tastes (40 columns with answer number) and another set of 50M cookies about web activities (columns are all continuous). Some of these 50M can be connected to ...
1
vote
1answer
21 views

Tool for clustering and cleansing data set

I have a large-ish data set (400K records) composed of two fields (both strings). I am looking for a tool that will enable me to cluster the data e.g. around the first column, either using exact ...
1
vote
1answer
17 views

In Affinity Propagation do the cluster centres minimize mean distance to all other points in the cluster?

For Affinity Propagation, do the cluster centers minimize the mean distance to all other points in the cluster?
0
votes
1answer
12 views

How do we visualize data in hierarchical clustering?

Can anybody tell me how to do visualization when applying hierarchical clustering to data with more than 2 features? Do we need to do dimensionality reduction before each clustering?
0
votes
0answers
20 views

Unsupervised learning/ clustering for data with multiple categorical variables

Dataset: I have been trying unsupervised clustering algorithms (K-modes & SOM) to cluster the students based on their grades in 3 exams. Should I one-hot encode the data (even though grades are ...
1
vote
1answer
14 views

Sensorfusion: Generate virtual sensor based on analysis of sensorsdata

I have a steam engine which is equipped with the following sensors: temperature sensor in the boiler room temperature sensor in the heating room pressure sensor in the boiler room rotations-per-...
0
votes
1answer
23 views

How to club the orders in such a way that maximum number of items are common amongst them?

Consider the following data set: The above table shows the quantity of each item used in the orders SO1 SO2 etc. I need to club the orders in such a way that maximum number of items are common amongst ...
1
vote
1answer
17 views

Should a Cluster Validity Index contain the same measure(s) as the Clustering Algorithm?

I'm currently trying to use cluster analysis as a tool for time-series aggregation for a project of mine. The dataset is high-dimensional (386-d), so no chance in assessing the cluster validity ...
0
votes
3answers
24 views

K-modes clustering: Estimating which features were most impactful on clustering?

I have entirely categorical data (survey results from users), so I've used k-modes clustering to better understand my users. I'm not an expert at clustering methods at all. Is there a way to known ...
1
vote
2answers
24 views

Is there any advantage in using Particle Swarm Optimization for clustering than K-Means?

I have read some paper about using particle swarm optimization. It doesn't look give much different result than K-Means. I tried to use PSO for clustering but the result is pretty much the same with K-...
1
vote
2answers
28 views

Record Linkage problem

I am building Matching Alogoritm using ML.Project is to match Internal customer data with external customer data.Features are names,address,city,state and zip. We create pairs between data sets and ...
1
vote
3answers
46 views

K means visualisation after reducing dimensionality with PCA

In clustering ($K$ means, for example) when I have $N$ features and after creating the model (with this $N$ features) to visualize this model I need to reduce this $N$ dimensions into $2$ or $3$ ...
0
votes
1answer
23 views

Clustering unbalanced dataset

The data I am working on has some really large price values and some really small values. What I did was first perform feature bagging on the data and got them labelled to (0,1) and then did ...
-1
votes
3answers
84 views

Is there any machine learning algorithm that can solve this problem?

I have a data set of 100000 samples with binary output. I would like to study the impact of Col_A (a continuous feature) on the output result. Col_A has values from 0 to 7000000 and when I add this ...
3
votes
1answer
59 views

Terminology: “flat geometry” in the context of clustering

Sklearn's documentation refers to "flat" or "non-flat" geometry of clusters to describe the use-cases of their implemented clustering algorithms. Those terms are not directly defined. However, the ...
3
votes
3answers
62 views

k-means classifies one data point as a group

I have 1000 sets of one dimensional data (360 each in length), and I want k means to classify what is a small/medium/large value (n_clusters=3) for each set of data, but I'm getting a lot of instances ...
0
votes
1answer
14 views

Clustering based on geolocation pair

I am trying to process a large set of location data where a list of start and end coordinate is given. For example, ...
0
votes
1answer
20 views

Clustering Metrics for large data

I have a dataset containing 150k rows and 10 columns. After clustering, I would like to get clustering metrics. Below are lists of metrics that I would like to use> ...
2
votes
1answer
64 views

clustering before or after PCA?

I'm newbie into data science, and I had some problems dealing with my project. I'm trying to visualize multidimensional data into 2D after clustering with using a lot of methods. (kmeans, DBSCAN, ...
0
votes
1answer
18 views

Clustering, Mixed Data Set with Ordinal and Nominal Scale Data

After reading a bit how categorical data can be considered in clustering, I came to the conclusion that most of the post do not make distinction between nominal scale data e.g. colour: red, green, ...
0
votes
1answer
21 views

Does orange transfrom categorial variables into dummy variables when using hierarchical clustering?

I am using Orange to cluster a large amount of data consisting of three attributes. Each attribute only contains categorical unordered variables. My question is does Orange transform categorical ...
0
votes
0answers
15 views

How does t-sne scale work? Is it possible to compare scales of two t-sne plots?

I have used t-sne for two convolutional layers of binary classifier (VGG16 like CNN). I want to compare them, but I am not sure if that is okay as the scales of these two outputs from t-SNE plots can ...
1
vote
1answer
98 views

Hierarchical clustering with precomputed cosine similarity matrix using scikit learn produces error

We want to use cosine similarity with hierarchical clustering and we have cosine similarities already calculated. In the sklearn.cluster.AgglomerativeClustering documentation it says: A distance ...
0
votes
0answers
17 views

Techniques for Collection/Graph Conversion to Cluster-able Data

Here are my problems; any techniques/papers as to how to approach this problem would be much appreciated. I also apologize for the vagueness of my question title; I do not really know if there is a ...
0
votes
1answer
27 views

clustering people according to answers on survey

Hi I am finding it hard to find online the best clustering algorithm for clustering people according to answers they gave on 20 question survey. There are four categories which each of these answers ...
0
votes
0answers
19 views

Unsupervised clustering improved with supervised classification accuracy

I have a set of labeled samples each containing up to 300 different objects. For every object I have a set of features describing the object. For example, Sample with label '1': 50 objects of type ...
2
votes
1answer
34 views

Clustering of Weekday Weekend Time Series Data

I have a dataset of the number of steps people take throughout a day over a period of months. I aggregated them so that each person will have an average weekday and weekend time series of steps. An ...