Questions tagged [clustering]

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than to those in other groups (clusters). It is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval etc.

Filter by
Sorted by
Tagged with
2
votes
3answers
1k views

Machine Learning with sometimes missing data

I'm trying to do an indoor locationing system based on my RSSI signal on my routers, I'm sniffing my network so I know what's the RSSI of my phone related to my routers antennas (I have 5 antennas all ...
3
votes
3answers
2k views

How cluster a twitter data-set?

I have a twitter data-set and I wanna extract their related topics. So, I decided to classify my Tweets into clusters using an unsupervised machine learning algorithm like k-means. This choice is made ...
0
votes
1answer
1k views

Orange, K-means clustering, memory error

I have a dataset of over 67 000 records, and I'm trying to run a k-means cluster analysis on that. Orange returns a memory error. The data is in an excel file, but I also tried to load it from a csv. ...
2
votes
1answer
161 views

Predicting and clustering at the same time?

I want to build a segmentation to substitute the existing RFM segmentation which is a basic segmentation based on the Recency, Frequency and Monetary values. The new segmentation will be used for two ...
8
votes
2answers
1k views

Fitting lines through large point clouds

I have a large set of points (order of 10k points) formed by particle tracks (movement in the xy plane in time filmed by a camera, so 3d - 256x256px and ca 3k frames in my example set) and noise. ...
3
votes
1answer
672 views

Derivation of Ward's formula for agglomerative clustering

Figure describe formula of ward method for hierarchical or agglomarative clustering i.e. increment of total error after merging two cluster. How did they get $$\frac{n_A\cdot n_B}{n_A+n_B} ||\vec ...
0
votes
1answer
101 views

Choosing the correct learning algorithm

I am kind of new to the data mining subject but i need help to choose a learning algorithm for my application: The problem: identifying that a certain curve or data set belongs to a certain fault in a ...
0
votes
1answer
181 views

Clustering of Temporal Data

I have been trying to find the correlation b/w the following type of temporal data for quite some time. DataSet: ...
1
vote
0answers
148 views

Clustering with restrictions - Silhouette and C index metrics

I am working on clustering with DBSCAN but with a certain constraint: the points inside a cluster have to be not only near in a Euclidean distance way but also near in a geographic distance way. It is ...
0
votes
2answers
287 views

How do I get latent features for a new row of data when doing non-negative matrix factorization?

Background: The basic set-up for non-negative matrix factorization (nmf), is that we take a matrix with non-negative elements, X and find two other non-negative matrices H and T such that $||X - HT|...
1
vote
0answers
46 views

Can cluster analysis of preclassified items gives idea about the classification performance?

Suppose in a classification we have a dataset with many features and their class, we want to select some features using which we can construct a classifier. We perform the cluster evaluation for the ...
3
votes
2answers
3k views

How can we evaluate DBSCAN parameters?

yes, DBSCAN parameters, and in particular the parameter eps (size of the epsilon neighborhood). In the documentation we have a "Look for the knee in the plot". Fine, but it requires a visual analysis....
0
votes
1answer
212 views

NNDSVD to initialize Convex-NMF

I'm working with the Convex Nonnegative Matrix Factorization Algorithm described in Ding, Li, Jordan 2008 ("Convex and Semi-Nonnegative Matrix Factorizations"). Good initialization strategies make ...
1
vote
2answers
54 views

Determining correlated product categories using store purchase history

I have a large dataset that contains product purchase history, like so: ...
0
votes
2answers
6k views

What is the relationship between clustering and association rule mining?

Cluster analysis or clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense or another) to each other than ...
0
votes
3answers
228 views

is this a classification or clustering problem?

Suppose, I am building a hotel recommendation system, that learns user profile based on his/her interaction with the system. I have two classes, "Like" and "Dislike". For example, a user likes 5 "5-...
6
votes
4answers
2k views

Why does OPTICS use the core-distance as a minimum for the reachability distance?

The OPTICS clustering algorithm defines $$\text{core-dist}_{\varepsilon,MinPts}(p)=\begin{cases}\text{UNDEFINED} & \text{if } |N_\varepsilon(p)| < MinPts\\ MinPts\text{-th smallest distance to ...
3
votes
1answer
152 views

How do i cluster binarized categorial data, without knowing the number of clusters?

I have a dataset of categorical data, and I need to cluster it without knowing k. I know algos for clustering data without knowing the number of centroids, like G-mean, but none works for categorial ...
2
votes
1answer
903 views

How do I obtain the weight and variance of a k-means cluster?

I am trying to reproduce the results of this paper, but using python and the HMMlearn library instead of matlab. The paper describes a procedure for using HMM (Hidden Markov Model) in order to predict ...
1
vote
1answer
107 views

Prepping Data For Usage Clustering

Dataset: I'm given the number of minutes individual customers use a product each day and am trying to cluster this data in order to find common usage patterns. My question: How can I format the data ...
2
votes
1answer
245 views

Clustering : handling categorical data, should we pivot and scale?

I'm a SQLServer DBA and in the new version of this tool, there are new features to integrate R scripts and use it easily with the DB objects. That sounds cool. But to use that, we have to know a ...
3
votes
3answers
1k views

Mahalanobis distance between two clusters

I want to calculate the Mahalanobis distance between cluster $a$ and cluster $b$, each consisting from a set of multidimensional points. Assuming no correlation, calculating the distance between a ...
0
votes
2answers
611 views

Hierarchical Clustering customized Linkage function

In my clustering project, I need to customize the linkage function, so that after each cluster merging I can update the inter-cluster distance in my own way. Currently I'm using scikit-learn ...
1
vote
3answers
118 views

Question about graphing the clusters in K means

I've used K means to cluster my data. Before using K means, I had used StandardScaler on my data to standardize the data. Now, I'm wondering how can I show the clusters of the original data. Scikit-...
1
vote
1answer
168 views

Question about the Silhouette number for K means clustering

I have used K means clustering. In order to find the best value for K, I've looked at the changes of inertia value vs K and also changes of average Silhouette number vs K. The graph for inertia seems ...
4
votes
2answers
113 views

clustering plus linear model versus non linear (tree) model

a team has to create models that predict the cost of deploying a machine over time. This is a regression problem. The team is further divided into two groups, A and B. Group A puts lots of ...
1
vote
1answer
92 views

Does it make sense to apply clustering on aggregation of data?

I was wondering if it makes sense to apply clustering techniques on an aggregation of data, like, I have three different sources of data such as S1 S2 and S3 where each of these sources share some ...
5
votes
1answer
328 views

Clustering efficiency in a discrete time-series

Is it possible to identify the point in time where the cluster separation is at its most in a discrete time series clustering? Say I have 4 clusters of discrete time series and I want to pick a ...
7
votes
4answers
335 views

What methods exist for distance calculation in clustering? when we should use each of them?

What methods exist for distance calculation in clustering? like Manhattan, Euclidean, etc.? Plus, I don't know when I should use them. I always use Euclidean distance.
4
votes
1answer
1k views

How to explain the outcome of k-means clustering?

I am currently conducting some analysis using NTSB aviation accident database. There are cause statements for most of the aviation incidents in this dataset that describe the factors lead to such ...
2
votes
1answer
68 views

How to detect a no good person in data?

I am struggling to figure a way to determine if a person is 'no good' or 'good'. A little bit about what I am trying to accomplish, I have a data set of a payment for a violation with a check or a ...
3
votes
3answers
2k views

Ensembling vs clustering in machine learning

Following the raise of ensembling (e.g ensembling of xgboost learners) after its recurrent wins in Kaggle competitions, using it has become a tradition in machine learning. However, some argue that ...
4
votes
1answer
221 views

Can I apply Clustering algorithms to the result of Manifold Visualization Methods?

Some methods related to manifold-learning are commonly stated as good-for-visualization, such as T-SNE and self-organizing-maps (SOM). I understand that when referring specifically to "visualization" ...
5
votes
1answer
3k views

Determinate K in K-Means Clustering

I have salary data of several user (Python list). Now I am using KMeans to cluster them. Given this data, Is there a way to figure out the best value for 'K' automatically through program? I tried ...
5
votes
4answers
393 views

Categorical Clustering of Users Reading Habits

I have a data set with a set of users and a history of documents they have read, all the documents have metadata attributes (think topic, country, author) associated with them. I want to cluster the ...
1
vote
3answers
1k views

K Means giving poor results

I have several user names and their salaries. Now I need to cluster user based on their salaries. I am using KMeans clustering and following is my code ...
0
votes
1answer
25 views

Combinatoric system which using in bookies “S3” system

I am interested what type of combinatorics is using for following bookmakers system called "S3": We have N={1..8} events We build ...
1
vote
0answers
164 views

Modelling latent online shopping states from e-commerce website clickstream data - What type of method to use?

I am new to Data Science but really enthusiastic about it. Really appreciate communities like this one, so thanks in anticipation. I entered a competition to solve a data mining challenge for an e-...
2
votes
1answer
2k views

Can KMeans clustering be used on word2vec output? [closed]

I have a dataset that has been trained on word2vec. Is it a good idea to cluster the output vectors?.
1
vote
0answers
283 views

Numerical data and different algorithms

I was hoping to get some help on a task I have been given. I have a data set with the chemical breakdown of a drink. My job is to find some sort of analysis with the given data. The data has 10 ...
2
votes
3answers
526 views

Clustering Crime Data which has {latitute, longitude, crime-type} tuples

I have a data set which has thousands of rows of {latitute, longitude, crime-type} tuples. Sample Data: ...
5
votes
3answers
156 views

Predictive clustering

I have an hypothesis but i don't know if it's true. If the cluster is dense and we apply a supervised learning on this data, the model generated by this cluster will be more efficient for new data ...
2
votes
0answers
78 views

Accept any suggestion to create training data from correlation matrix to find odd one out to identify difference in variation

I have N time varying feature vectors obtained by recording different parameters over time.This results in N*N similarity matrix which contains one to one correlations value for each feature. We need ...
6
votes
2answers
582 views

Is this cluster analysis / prediction?

I have a series of seemingly random data dripping in one value at a time through time. Although it appears to be random, the data forms clusters when certain attributes are analysed which the charts ...
4
votes
2answers
57 views

How to build a mean prototype from data

I have a dataset with physiological measures of subjects along time. I would like to create (or select) a mean prototype example in order to be able to identify in new examples how far are they from ...
2
votes
0answers
90 views

Selecting the number of hashes for minhash? Working with extremely sparse data and want more collisions

I'm attempting to use minhash to generate clusters and similarities, and I am primarily using ideas from these resources. http://www2007.org/papers/paper570.pdf https://chrisjmccormick.wordpress.com/...
1
vote
2answers
689 views

Can i find similar players using a clustering method like the k-mean algorithm?

I am working on a data mining project on NBA data. I want to make a recommendation system similar to the google one, where you search for players and you get recommendation for similar players. I ...
3
votes
1answer
134 views

Why does image segmentation benefit from fuzzy clustering?

In many image processing papers, I've seen that they used fuzzy logic for segmentation I wonder how fuzzification impact the result that made Fuzzy-C-Means better than ordinary K-Means. PS. If ...
2
votes
0answers
80 views

Finding dominating attributes with in the clusters generated

I am having a dataset of customers where each customer is represented as some feature vector and I am applying K-means algorithm to this dataset. On the basis of those features, I can abstract and ...
0
votes
1answer
702 views

How to retrieve the clustering results of rpart

I am using rpart package in order to create a segmentation of my data using decision tree. As final result I want to obtain a classification of my data. For exemple, if the rpart devide data into 3 ...