Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
3
votes
4answers
168 views

Definition of “inside” in K-means?

After conducting a cluster analysis using K-means, I have new data coming online that I need to detect anomalies with. Anomalies are assumed to not be within the clusters. So, how is one to define "...
2
votes
1answer
87 views

Spatial clustering based on response to inputs and building a reduced model

Stats version: I have a few measurements of a function that takes three inputs and produces a few 2D fields of outputs: $f_i(a,b,c;x,y)$, with $f$ being a vector of several quantities. I would like to ...
5
votes
2answers
645 views

How are clusters from DBSCAN sometimes non-convex?

I've been using clustering in my bag of ML techniques for quite some time now, and I've never found a satisfying answer to this question. In DBSCAN, we define a maximum radius with which to form ...
2
votes
1answer
90 views

Joint spatial clustering: How to force clusters to minimally contain datapoints from all datasets

I want to jointly cluster datapoints coming from different datasets (50 datasets with around 2000 points each). I would like to then extract information associated to the datapoints belonging to the ...
1
vote
3answers
108 views

Clustering uncertain data with independent uncertainty per dimension

I have $n$ objects located in a $d$ dimensional space, however I do not know their exact coordinates. For each object and each dimension, I have a set of noisy measurements of the coordinate. I would ...
2
votes
2answers
280 views

Kmeans on mixed dataset with high level for categ

My retail dataset contains 3 numeric attributes and two categorical attributes Time and ID with 50,000 records. Both categorical attributes have more than 20 thousand levels and their format is 1/11/...
3
votes
1answer
169 views

User profiling with Mahout from categorized user behavior

I'm trying to cluster and classify users with Mahout. At the moment I am at the planning phase, my mind is completely mixed with ideas, and since I'm relatively new to the area I'm stuck at the data ...
4
votes
1answer
269 views

Can you use clustering to pick out signals in noisy data?

As my first project into data science, I would like to pick out the main clusters in noisy data. I think a good example would be trying to pick out certain links on a given StackExchange question that ...
1
vote
0answers
91 views

SPSS style custom tables in Orange

Is there a way to draw custom tables like in SPSS where you put some attributes to rows and some to column to visualize clustering results?
8
votes
3answers
1k views

Algorithm for segmentation of sequence data

I have a large sequence of vectors of length N. I need some unsupervised learning algorithm to divide these vectors into M segments. For example: K-means is not suitable, because it puts similar ...
5
votes
1answer
183 views

how can I generate a Bernoulli block mixture model in matlab?

I am trying to write the code of a Bernoulli block mixture model in matlab, but am facing an error every time I run the function. In particular, I'm having a problem with how to relate the ...
12
votes
1answer
1k views

MinHashing vs SimHashing

Suppose I have five sets I'd like to cluster. I understand that the SimHashing technique described here: https://moultano.wordpress.com/2010/01/21/simple-simhashing-3kbzhsxyg4467-6/ could yield ...
5
votes
2answers
689 views

Statistical distances for time series of distributions

I am interested in clustering $N$ time series of $T$ 'values' each. These values are distributions (which can be represented by their cumulative distribution functions (cdf), or their probability ...
2
votes
1answer
170 views

What kind of analysis can I do in this dataset? [closed]

I have a dataset of clients (their city, name, age, gender, number of children) and another dataset about the products that they have bought. i have been asked to do: extract knowledge about client ...
3
votes
1answer
205 views

Brown clustering, graph partitioning, agglomerative clustering - libraries/software

I need to do some experimenting with Brown clustering, graph partitioning, agglomerative clustering. 1) Are there Python/Matlab libraries for that? I know sklearn.cluster but it doesn't have ...
4
votes
4answers
141 views

Algorithm for deriving mutiple clusters

Suppose I have a set of data(with 2 diemensional feature space), and I want to obtain clusters from them. But I do not know how many clusters will be formed. Yet I want separate clusters(The number ...
5
votes
2answers
142 views

How to cluster a link traversal dataset

I'm using Google Analytics on my mobile app to see how different users use the app. I draw a path based on the pages they move to. Given a list of paths for say a 100 users, how do I go about ...
14
votes
2answers
10k views

Using attributes to classify/cluster user profiles

I have a dataset of users purchasing products from a website. The attributes I have are user id, region(state) of the user, the categories id of product, keywords id of product, keywords id of ...
3
votes
1answer
370 views

How do I cluster data that is a mix of text & categorical data? [closed]

I have a set of strings, each also has soem categorical information associated with it. The categorical information isn't always great though, so I need to cluster the messages based on the text ...
14
votes
2answers
2k views

Fast k-means like algorithm for 10^10 points?

I am looking to do k-means clustering on a set of 10-dimensional points. The catch: there are 10^10 points. I am looking for just the center and size of the largest clusters (let's say 10 to 100 ...
3
votes
1answer
435 views

Finding user similarities within informal data sets

I'm new to all this and am putting together a learning project. I've decided on finding similarities between users in a data set such as http://en.wikipedia.org/wiki/Enron_Corpus. After doing a bit of ...
4
votes
1answer
622 views

NoSQL engine/service recommendation for geolocation data

First of all, I am new in this field we call big data, so my questions may be naive. In order to build an application, which deals with geolocation data, which could be : latitude and longitude ...
17
votes
4answers
18k views

K-means: What are some good ways to choose an efficient set of initial centroids?

When a random initialization of centroids is used, different runs of K-means produce different total SSEs. And it is crucial in the performance of the algorithm. What are some effective approaches ...
0
votes
2answers
299 views

Tidying Time Intervals for Plotting a Histogram in R

I'm doing some cluster analysis on the MLTobs from the LifeTables package and have come across a tricky problem plotting ...
3
votes
2answers
527 views

Using clustering and Lasso with cv

I used clustering on my dataset. Now when I'm trying to use a LASSO with cv to predict a response, one of the variables it takes into consideration is which cluster a new point is classified into.(I ...
2
votes
1answer
1k views

Optimizing Weka for large data sets

First of all, I hope I'm in the right StackExchange here. If not, apologies! I'm currently working with huge amounts of feature-value vectors. There are millions of these vectors (up to 20 million ...
1
vote
2answers
254 views

Complete link clustering

I'm conjecturing that with Complete-linkage clustering two elements from the same cluster will always be closer to each other some other element from another cluster. In more formal terms: Let $C$ ...
0
votes
2answers
279 views

about predict the class for a new datapoint

I have a new data point and want to classify it into the existing classes.I can calculate pairwise distance for the new point to all existing points(in the existing classes). I know using KNN would be ...
2
votes
3answers
459 views

Data mining- Clustering techniques

I have a project for comparison between clustering techniques using the data set of SSA for birth names from 1910-2013 years for the different states. I have finished applying my clustering ...
1
vote
1answer
869 views

Predictive Model for Customer Payment Pattern

I am working for a logistics firm and there are approx. 750+ customers who avail our services. I am in the process of building and generating some insight for the business based on the payments made ...
2
votes
1answer
200 views

Clustering not producing even clusters

I'm using k-means clustering to processes running on machines. Dataset sample : machine name, process m1,java m2,tomcat m1,word m3,excel Build a matrix of ...
1
vote
1answer
693 views

Measuring similarity for sets with same cardinality

The Jaccard coefficient measures similarity between finite sample sets, and is defined as the size of the intersection divided by the size of the union of the sample sets. I had 100 sets all of same ...
2
votes
2answers
2k views

Probability of similarity of two clusters

For example, there are 4 objects. Each object has some elements. There are 3 clusters. Each element of each object belongs to some cluster. Clustering was done before, and we know only how many ...
1
vote
2answers
1k views

How do I get Twitter Dataset for Visualization [closed]

I am a newbie. I would like to do visualization on twitter data : top trends based on country (over map) and time variations (for each months in 1 year or for each year) . Can someone tell me where ...
3
votes
2answers
320 views

Graph modularity measure

There are several metrics for the quality of a graph clustering, e.g. Newman modularity. These enable you to compare two candidate clusterings of the same graph. Does anyone know a metric that will ...
1
vote
1answer
105 views

Assigning new items to existing similarity based clustering

Given some clusters created from similarity measures between items, is there a recommended way to assign a new item to an existing cluster based on similarity alone? (i.e. avoiding re-clustering) ...
4
votes
3answers
1k views

Partitioning Weighted Undirected Graph

I have an array of edges and weights: [['a', 'b', 4], ['a', 'c', 3], ['c', 'a', 2], ...] I have about 100,000 edges and weights are between 1 and 700, most ...
7
votes
1answer
1k views

How to extract features and classify alert emails coming from monitoring tools into proper category?

My company provides managed services to a lot of its clients. Our customers typically uses following monitoring tools to monitor their servers/webapps: OpsView Nagios Pingdom Custom shell scripts ...
22
votes
2answers
8k views

How to deal with time series which change in seasonality or other patterns?

Background I'm working on a time series data set of energy meter readings. The length of the series varies by meter - for some I have several years, others only a few months, etc. Many display ...
1
vote
1answer
499 views

Assign new point to a class using spectral clustering

Say I used spectral clustering to cluster a data-set $D$ of points $X_0 - X_n$ into a number $C$ of clusters. How can I efficiently assign a new single point $X_{n+1}$ to his convenient cluster? Do I ...
6
votes
5answers
616 views

What would be a good way to use clustering for outlier detection?

For simplicity let's assume the feature space is the XY plane.
11
votes
4answers
456 views

Using Clustering in text processing

Hi this is my first question in the Data Science stack. I want to create an algorithm for text classification. Suppose i have a large set of text and articles. Lets say around 5000 plain texts. I ...
4
votes
1answer
424 views

Can some one explain how PCA is relevant in extracting parameters of Gaussian Mixture Models

I am having some difficulty in seeing connection between PCA on second order moment matrix in estimating parameters of Gaussian Mixture Models. Can anyone connect the above??
10
votes
3answers
3k views

Log file analysis: extracting information part from value part

I'm trying to build a data set on several log files of one of our products. The different log files have their own layout and own content; I successfully grouped them together, only one step ...
1
vote
0answers
251 views

Analyze paragraphs using Neuroph

Currently we are regularly analyzing sets of paragraphs every month. I would like to automate this and split each paragraphs into chunks of data. To do this I would like to employ a neural network. ...
9
votes
2answers
11k views

Clustering of documents using the topics derived from Latent Dirichlet Allocation

I want to use Latent Dirichlet Allocation for a project and I am using Python with the gensim library. After finding the topics I would like to cluster the documents using an algorithm such as k-means(...
3
votes
1answer
98 views

Analyzing mobile usage. What kind of approach should I apply?

I need to analyse a dataset about mobile phone usage (#calls, #sms, #internetConnections) per each cell and hour in the different days. ...
2
votes
4answers
6k views

Small project ideas for Machine Learning [closed]

I need some serious help. I am supposed to implement a project (Non-Existing as of now) for my Machine Learning course. I have no basics in AI or Data mining or Machine learning. I have been searching ...
3
votes
1answer
4k views

R Script to generate random dataset in 2d space

I want to analyze the effectiveness and efficiency of kernel methods for which I would require 3 different data-set in 2 dimensional space for each of the following cases: BAD_kmeans: The data set ...
5
votes
2answers
904 views

What is the best Data Mining algorithm for prediction based on a single variable?

I have a variable whose value I would like to predict, and I would like to use only one variable as predictor. For instance, predict traffic density based on weather. Initially, I thought about using ...