Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
1
vote
1answer
60 views

Machine learning tasks classification

I am trying to be precise in definitions. We can solve regression, classification, clusterisation, dimensionality reduction, visualization, feature extraction tasks. But also there are supervised, ...
0
votes
1answer
57 views

Creating better features for clustering

I am trying clustering for the first time trying to separate my user into three categories (or the categories I though that they will should fall in). First of all I have two tables that describe ...
0
votes
0answers
58 views

Cluster method with binary variable

I need to do a cluster analysis for the following variables: ...
0
votes
1answer
13 views

How to assure the best cluster number for the given instances?

The dataset has 212,534 instances and every instance has 128 dimensions. I want to build a cluster model on it. Firstly I should select the best cluster number by literally calculate the metrics for ...
-1
votes
1answer
61 views

How can I get decision tree like rules for my cluster(s)

After performing clustering and detailed cluster analysis, I am confident that my clusters make sense. Now, for each cluster I would like to generate rules in the form of decision tree output. With ...
1
vote
0answers
284 views

“Memory Error” - Kmeans in python using pandas DataFrame

I am trying to predict on my "dataset_to_predict" having size of (297000 x 5120). While Memory usage is under 50%. No Specific Error message. I'm trying to find # of k using elbow method - Got ...
2
votes
2answers
45 views

Clustering of news combining headline and main article

I want to classify German police news articles and do an automated classification/clustering with regards to the kind of crime committed. Thus far I am not getting great results. Often times the ...
1
vote
0answers
22 views

Can I use entropy as a measure for determining significant variables in a cluster after clustering?

After clustering my data into k groups, I would like to determine for each of the clusters, which dimensions(variables) significantly describe that particular cluster. For example, lets say cluster A ...
-1
votes
1answer
35 views

Looking for an algorithm which does Max Sum Clustering

I have a very limited background in data science and dataset processing and I was hoping I could get some help here. I am doing some work that requires clustering certain data points having $(x, y)$ ...
-1
votes
1answer
32 views

How can I categoriese / classify a cluster of words?

I am just wondering if it is possible to classify word clusters? For example if I provide you an array of words [bird,chicken,dock,park,apple,grapes,furits,juice] ...
0
votes
1answer
26 views

A bit confused regarding clustering of users in a dataset

I have a dataset of book reviews: ...
-1
votes
1answer
144 views

No target variable in my data

I have a list of transactions of bus route from place to place, I don't have any target variable here. I was asked to give meaningful insights from the data, what can I do here? I cleaned the data, ...
0
votes
1answer
41 views

Preparing dataframe to carry k-means clustering [closed]

Im trying to apply 3 different algorithms of clustering on my dataset. to check which one fits the best. I'm confused how should I convert my dataframe -k-means -DBSCAN -hierarchical clustering ...
0
votes
0answers
87 views

sklearn & Meanshift for NLP only returns 1 cluster

I am using sklearn.clustering to work with some text data and the MeanShift algorithm. I have: Done all standard NLP data prep like lemmatizing, removing stop ...
2
votes
1answer
41 views

Conditional Entropy and Mutual Information - Clustering evaluation

First of all, I am doing clustering and I have the true labels for my data. For evaluation, I am using the weighted average of the entropy values for each predicted cluster. I also came across with ...
2
votes
1answer
35 views

how to implement a hierarchical clustering technique using parallel execution in R

In R, currently to implement wards method for hierarchical clustering, I use the following code - results <- hclust(data, "ward.D2"). However as my data size has ...
3
votes
2answers
199 views

Random-Forest-based Similarity Matrix for clustering: how does it behave?

I am in the following context: Data: static, baseline health data at the patient level, 40 features, sparse (~ 25 binary features with many 0 or many 1 + other categorical features) Objective : to ...
6
votes
3answers
293 views

How to give a higher importance to certain features in a (k-means) clustering model?

I am clustering data with numeric and categorical variables. To process the categorical variables for the cluster model, I create dummy variables. However, I feel like this results in a higher ...
0
votes
0answers
24 views

How to analyze sets within Orange

Hi coming up with similar question like Q49352 - Clustering with sets as values . I am looking for the right way to work with sets in Orange. My set has a list of workflow steps from a workflow ...
0
votes
0answers
25 views

Is the mean shift algorithm adapted to my problem?

I'm currently building a model that can detect abnormalities in a timeserie. First, we predict the next steps and then we compare the prediction with what we measure in real time. We want to see if ...
0
votes
1answer
27 views

How to cluster text-based software requirements

I'm beginner in deep learning and I'd like to cluster text-based software requirements by themes (words similarities/frequency of words) using neural networks. Is there any example/tutorial/github ...
-2
votes
1answer
48 views

Clustering for variables with large amount of categories

I have a dataset which, has variables with a lot of categories (some more than 1000). Since, large amount of categories effect the accuracy of the model. I saw some literature stating that if you do ...
0
votes
0answers
16 views

Why is my LOF algorithm producing the opposite result it should?

What could cause the local outlier factor (LOF) to output below 1.0 for outliers and above 1.0 for inliers? I have my code sort of working just by inverting the output, but I can't figure out what's ...
3
votes
2answers
5k views

Perform k-means clustering over multiple columns

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not ...
1
vote
1answer
59 views
0
votes
0answers
258 views

How to validate clusters after calculating Gower distances and Ward's clustering in R

I am trying to apply Ward's clustering on a mixed types dataset, and wanna explain what I did (maybe helpful to others), and I have some questions regarding this analysis, mainly how to validate my ...
2
votes
2answers
61 views

How long would it take to become proficient in machine learning for someone with a non-statistical mathematical background?

I am currently a postdoc and my PhD was in applied mathematics in the area of numerical analysis and electromagnetic/acoustic wave propagation. There was no statistical element to my PhD, it was ...
0
votes
0answers
340 views

Clustering of multi-label data

The dataset consists of 1) a set of objects and 2) a set of labels, which are used to describe the objects. For the moment, for simplicity sake, each label can be marked as either true or false (...
1
vote
2answers
55 views

Is there any clustering algorithm to find longest continuous subsequences?

I have data which contains access duration of some items. Example: t0~t5 is the access time duration, 1 means the items was accessed in the time duration, 0 means it wasn't. ...
1
vote
4answers
94 views

How do I identify clusters that match on categorical data?

I am seeking some directions for a proper path to research the solve for this problem: My company made all our employees take a "StrengthFinders" test, which results in every employee being assigned ...
3
votes
1answer
37 views

How to approach clustering of time-series with a single variable

Let me preface this by saying that I'm a complete beginner to R and data science in general, so my apologies if this is a rather trivial question. I do have a rough idea of what I would like to ...
0
votes
1answer
268 views

How to run AgglomerativeClustering on a big data in python?

I run AgglomerativeClustering on a sample of data and fit a model. then I decide to predict this fit for all of my data but I got MemoryError. How can I run ...
1
vote
1answer
637 views

Building a large distance matrix

I am trying to build a distance matrix for around 600,000 locations for which I have the latitudes and longitudes. I want to use this distance matrix for agglomerative clustering. Since this is a ...
1
vote
0answers
444 views

Are there any public datasets about mental health of patients containing physiological and psychological symptoms?

I would like to segment mental illnesses with clustering using machine learning. To do so I need training dataset which contains physiological and psychological symptoms of an subject. The closest ...
1
vote
1answer
2k views

Clustering based on distance between points [closed]

I am trying to cluster geographical locations in such a way that all the locations inside each cluster are at max within 25 miles of each other. For this, I am using Agglomerative clustering. I am ...
0
votes
2answers
203 views

Variable Importance in unsupervised anomaly detection algorithms

I am working on an anomaly detection problem to detect fraud in insurance claims. I have used the PyOD package and used algorithms like ABOD, CBLOF, Isolation Forest, and AutoEncoder. I couldn't find ...
2
votes
2answers
133 views

Is it wrong if I cluster numerical attributes and categorical attributes separately?

I have a dataset of credit customers containing mixed data types (numerical and categorical with several levels). I am trying to perform segmentation so that I can end up with k groups and then build ...
0
votes
2answers
420 views

k modes: optimal k

I have categorical data and I'm trying to implement k-modes using the GitHub package available here. I am trying to create clusters in my (large) dataset of say, 5-7 records, each of most similar ...
1
vote
1answer
99 views

What does Make Density Based Clusterer in Weka do?

In Weka, there is a clustering algorithm with the name as Make Density Based Clusterer. When going through its properties, it takes a clusterer as base clusterer(I took it as K-means with k=3). It ...
1
vote
1answer
143 views

How to calculate a weighted Hierarchical clustering in Orange

I am doing my first cluster analysis with Orange (which I recently discovered and looks promising for this iterative and interactive process). Apparently, there are several methods of creating ...
1
vote
2answers
286 views

K-modes implementation in pyspark

I'm looking for an implementation of k-modes in pyspark. I found this and this as implementations. First, I tried implementing k-modes using the first link and faced issues. So I went ahead and tried ...
1
vote
1answer
35 views

Define value of a centroid

I want to choose a clustering algorithm for which I can define the value of centroids (and not only the number of them). Which algorithm should I look at? My understanding of k-means is that I can ...
1
vote
0answers
69 views

Best classification technique for following kind of data set

I have a large table where each record or row represents a single salesperson, and there are 50 columns or dimensions where each column represents one of 50 products potentially sold by any given ...
3
votes
1answer
35 views

Cluster Analysis - Comparing Same Individuals Clustered Across Different Datasets with different features

I have an interesting problem, and I think my Google is failing me since I can't find the same problem anywhere. I have a set of individuals. I have 4 different datasets, with (some) to (all) of ...
2
votes
3answers
77 views

Are there algorithms for clustering objects with pairwise distances, without computing all pairwise distances?

I'm looking for a clustering algorithm that clusters objects, by using their pairwise distances, without needing to calculate all pairwise distances. Normally pairwise clustering is done like this: (...
0
votes
1answer
87 views

Clustering by Distributions of Groups of observations

I have data as shown below. Like the groups shown A,B,C.. there are roughly 5400 groups of data and 15 observations for every group (I sampled 6-7 for each group and pasted them here). ...
1
vote
0answers
32 views

Clustering and graphing similarities of sentence subjects

I have a bunch of sentences. Each sentence is given a weight of how "close" it is to a particular subject. Ex. "I love reading math books" Subjects for the above sentence = ...
-1
votes
1answer
47 views

Why in spectral clustering number of eigen vectors is same as number of clusters that we want?

In spectral clustering we take eigenvector corresponding to K smallest eigenvalues. Then we do K means clustering on these eigenvector to get final clusters. What will happen if we take different ...
0
votes
1answer
167 views

Use output of K-Mean for Logistics regression

I've created a binary classifier using K Mean, which predicts fraud and legitimate accounts, 0 and 1. This uses two features, let's say, A and B. Now, I want to use other features like C and D, to ...
1
vote
1answer
134 views

Hierarchical Clustering and Variable Selection

I am using "Single linkage" hierarchical algorithm to cluster my data points with Gower Distance as my data have both qualitative and quantitative variables. After applying this for the full ...