Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
8
votes
4answers
18k views

Clustering for mixed numeric and nominal discrete data

My data includes survey responses that are binary (numeric) and nominal / categorical. All responses are discrete and at individuals level. Data is of shape (n=7219, p=105). Couple things: I am ...
3
votes
2answers
77 views

How do I learn experimental methodology? When is it relevant?

I just graduated in Computer Science, with a very theoretical background but without any kind of Data Science or Artificial Intelligence experience, and I working on my own to discover those two ...
1
vote
0answers
159 views

What are references explaining Hugo Steinhaus early “data science” work?

Historical background: Hugo Steinhaus can be considered as an early father of data science. He authored the paper Sur la division des corps matériels en parties, Bulletin de l’Académie Polonaise des ...
0
votes
1answer
131 views

Agglomerative Clustering Stopping Criteria

I am trying to implement section 3.4 of paper Predicting Important Objects for Egocentric Video Summarization where they have created a distance matrix of frame histograms. In short, let say ...
1
vote
1answer
208 views

Attribute discretization

I have a dataset where I need to explore using unsupervised technics (clustering and association rules). What are the best strategies to discretize the numeric attributes? Also, does this (attribute ...
1
vote
0answers
288 views

tf-idf clustering

I have over a million text documents that I would like to cluster. I used tf-idf modeling and term vector cosine for identifying similar documents in the corpus, which appeared to work well. Some ...
0
votes
1answer
68 views

Clustering related areas with k-means

I am trying to cluster related areas of knowledge based in publications.For example a researcher has 3 keywords in a paper, in another paper he has 5 keywords, but 3 keywords are the same in the both ...
2
votes
1answer
101 views

Clustering with constraints

I have a dataset of this form: chrX posX labelX where chrX refers to the chromosome number, ...
2
votes
1answer
411 views

How should clickstream data be prepared before user segmentation can be performed?

I'm interested in doing segmentation/clustering of users in clickstream data and am looking for some good suggestions about how to go about it. Lets say my data consists of observations made up of ...
1
vote
1answer
104 views

Devices behavior in one continuous variable vs events rate

I have devices on which I have time series data of one continuous variable. I have to evaluate the relation between the profile of that variable on those devices and "events". Those events are given ...
4
votes
1answer
160 views

How to create clusters of position data?

I am asking this question because the previous one wasn't very helpful and I asked about a different solution for the same problem. The Problem I have lateral positions, ...
-1
votes
1answer
181 views

Clustering large number of strings based on tags

I have string representations of text written by users in the form of parts of speech tags like so: $NNDN,OVDANPN,PNVRV,^^V,^^!$^OV and ...
7
votes
0answers
124 views

Is Minimax Linkage a Lance-Williams hierarchical clustering?

I found the following article on "Hierarchical Clustering With Prototypes via Minimax Linkage". It is stated in Property 6 that Minimax linkage cannot be written using Lance–Williams updates. A ...
4
votes
1answer
161 views

Classification problem where one attribute is a vector

Hello I am a layman trying to analyze game data from League of Legends, specifically looking at predicting the win rate for a given champion given an item build. Outline A player can own up to 6 ...
2
votes
2answers
1k views

How to rank documents using Bag of words approach

I want to cluster the documents I get for Google scholar search using the Bag of words model. I thought of using Java as the language. Assume for the keyword k, Google scholar gives me 50 results. If ...
1
vote
2answers
45 views

Best way to fuse clusters smaller than a certain size in a distributed way?

I have X clusters of size (n1,n2...,nX) and I want to fuse small clusters with less than T members with the nearest adjacent cluster i.e. the nearest neighboring cluster. The problem is when I've got ...
5
votes
1answer
607 views

Cluster directed graph into DAG

I am looking for an algorithm that clusters a directed graph into a set of clusters that form a directed acyclic graph. For example, given: nodes: {A, B, C, D} with edges: {(A,B), (B,A), (A,C), (C,D),...
3
votes
2answers
3k views

How to convert vector values to fit k-means algorithm function?

I have a set of user objects that I want to group using a $k$-means function from their quiz answers. Each quiz question had predefined answers with letter values "a", "b", "c", "d". If a user answers ...
4
votes
1answer
12k views

How to plot/visualize clusters in scikit-learn (sklearn)?

I have done some clustering and I would like to visualize the results. Here is the function I have written to plot my clusters: ...
24
votes
4answers
22k views

Is it necessary to standardize your data before clustering?

Is it necessary to standardize your data before cluster? In the example from scikit learn about DBSCAN, here they do this in the line: ...
2
votes
0answers
62 views

Spatial clustering of data points on a grid to obtain variable resolution map with constant statistical confidence

I have a grey valued image which is calculated as the mean of a series of images. The value of each pixel is therefore associated to a standard error. The pixel values and the relative standard error ...
1
vote
1answer
311 views

R: Comparing dissimilarity between metabolic models with discrete wavelet transformation

I’m working on comparing bacteria metabolic models. Each model has a set of metabolites (around 2000) and their concentration for 200 time points. I’m in the process of comparing the models to cluster ...
23
votes
3answers
17k views

K-means incoherent behaviour choosing K with Elbow method, BIC, variance explained and silhouette

I'm trying to cluster some vectors with 90 features with K-means. Since this algorithm asks me the number of clusters, I want to validate my choice with some nice math. I expect to have from 8 to 10 ...
3
votes
1answer
3k views

How to calculate most frequent value combinations

I have the following CSV data: ...
6
votes
3answers
3k views

How to evaluate clustering success in a completely unsupervised system?

The algorithm in question is Kohonen's SOM. But the question could also apply to PCA and some others. When the umatrix (or the codebook?) is examined, is there a way to tell how successful clustering ...
3
votes
4answers
168 views

Definition of “inside” in K-means?

After conducting a cluster analysis using K-means, I have new data coming online that I need to detect anomalies with. Anomalies are assumed to not be within the clusters. So, how is one to define "...
2
votes
1answer
87 views

Spatial clustering based on response to inputs and building a reduced model

Stats version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: $f_i(a,b,c;x,y)$, with $f$ being a vector of several quantities. I would like to ...
5
votes
2answers
642 views

How are clusters from DBSCAN sometimes non-convex?

I've been using clustering in my bag of ML techniques for quite some time now, and I've never found a satisfying answer to this question. In DBSCAN, we define a maximum radius with which to form ...
2
votes
1answer
90 views

Joint spatial clustering: How to force clusters to minimally contain datapoints from all datasets

I want to jointly cluster datapoints coming from different datasets (50 datasets with around 2000 points each). I would like to then extract information associated to the datapoints belonging to the ...
1
vote
3answers
107 views

Clustering uncertain data with independent uncertainty per dimension

I have $n$ objects located in a $d$ dimensional space, however I do not know their exact coordinates. For each object and each dimension, I have a set of noisy measurements of the coordinate. I would ...
2
votes
2answers
274 views

Kmeans on mixed dataset with high level for categ

My retail dataset contains 3 numeric attributes and two categorical attributes Time and ID with 50,000 records. Both categorical attributes have more than 20 thousand levels and their format is 1/11/...
3
votes
1answer
169 views

User profiling with Mahout from categorized user behavior

I'm trying to cluster and classify users with Mahout. At the moment I am at the planning phase, my mind is completely mixed with ideas, and since I'm relatively new to the area I'm stuck at the data ...
4
votes
1answer
266 views

Can you use clustering to pick out signals in noisy data?

As my first project into data science, I would like to pick out the main clusters in noisy data. I think a good example would be trying to pick out certain links on a given StackExchange question that ...
1
vote
0answers
91 views

SPSS style custom tables in Orange

Is there a way to draw custom tables like in SPSS where you put some attributes to rows and some to column to visualize clustering results?
8
votes
3answers
1k views

Algorithm for segmentation of sequence data

I have a large sequence of vectors of length N. I need some unsupervised learning algorithm to divide these vectors into M segments. For example: K-means is not suitable, because it puts similar ...
5
votes
1answer
181 views

how can I generate a Bernoulli block mixture model in matlab?

I am trying to write the code of a Bernoulli block mixture model in matlab, but am facing an error every time I run the function. In particular, I'm having a problem with how to relate the ...
12
votes
1answer
1k views

MinHashing vs SimHashing

Suppose I have five sets I'd like to cluster. I understand that the SimHashing technique described here: https://moultano.wordpress.com/2010/01/21/simple-simhashing-3kbzhsxyg4467-6/ could yield ...
5
votes
2answers
679 views

Statistical distances for time series of distributions

I am interested in clustering $N$ time series of $T$ 'values' each. These values are distributions (which can be represented by their cumulative distribution functions (cdf), or their probability ...
2
votes
1answer
169 views

What kind of analysis can I do in this dataset? [closed]

I have a dataset of clients (their city, name, age, gender, number of children) and another dataset about the products that they have bought. i have been asked to do: extract knowledge about client ...
3
votes
1answer
205 views

Brown clustering, graph partitioning, agglomerative clustering - libraries/software

I need to do some experimenting with Brown clustering, graph partitioning, agglomerative clustering. 1) Are there Python/Matlab libraries for that? I know sklearn.cluster but it doesn't have ...
4
votes
4answers
140 views

Algorithm for deriving mutiple clusters

Suppose I have a set of data(with 2 diemensional feature space), and I want to obtain clusters from them. But I do not know how many clusters will be formed. Yet I want separate clusters(The number ...
5
votes
2answers
142 views

How to cluster a link traversal dataset

I'm using Google Analytics on my mobile app to see how different users use the app. I draw a path based on the pages they move to. Given a list of paths for say a 100 users, how do I go about ...
14
votes
2answers
9k views

Using attributes to classify/cluster user profiles

I have a dataset of users purchasing products from a website. The attributes I have are user id, region(state) of the user, the categories id of product, keywords id of product, keywords id of ...
3
votes
1answer
364 views

How do I cluster data that is a mix of text & categorical data? [closed]

I have a set of strings, each also has soem categorical information associated with it. The categorical information isn't always great though, so I need to cluster the messages based on the text ...
14
votes
2answers
2k views

Fast k-means like algorithm for 10^10 points?

I am looking to do k-means clustering on a set of 10-dimensional points. The catch: there are 10^10 points. I am looking for just the center and size of the largest clusters (let's say 10 to 100 ...
3
votes
1answer
435 views

Finding user similarities within informal data sets

I'm new to all this and am putting together a learning project. I've decided on finding similarities between users in a data set such as http://en.wikipedia.org/wiki/Enron_Corpus. After doing a bit of ...
4
votes
1answer
621 views

NoSQL engine/service recommendation for geolocation data

First of all, I am new in this field we call big data, so my questions may be naive. In order to build an application, which deals with geolocation data, which could be : latitude and longitude ...
17
votes
4answers
17k views

K-means: What are some good ways to choose an efficient set of initial centroids?

When a random initialization of centroids is used, different runs of K-means produce different total SSEs. And it is crucial in the performance of the algorithm. What are some effective approaches ...
0
votes
2answers
297 views

Tidying Time Intervals for Plotting a Histogram in R

I'm doing some cluster analysis on the MLTobs from the LifeTables package and have come across a tricky problem plotting ...
3
votes
2answers
515 views

Using clustering and Lasso with cv

I used clustering on my dataset. Now when I'm trying to use a LASSO with cv to predict a response, one of the variables it takes into consideration is which cluster a new point is classified into.(I ...