Questions tagged [k-means]

k-means is a family of cluster analysis methods in which you specify the number of clusters you expect. This is as opposed to hierarchical cluster analysis methods.

Filter by
Sorted by
Tagged with
1
vote
1answer
27 views

How to improve results for clustering of words

I have a list of words (names actually) on which I would like to apply some entity resolution. My first guess was to create clusters of similar names so I could extract a representative entity from ...
0
votes
0answers
10 views

Can someone provide me the code of the MiLoF(Memory Efficient Local Outlier Factor) algorithm?

I have to code the MiLoF algorithm for detecting outliers in an unsupervised manner using non-stationary data. I am attaching the paper which explains the algorithm here. However, there are many ...
0
votes
1answer
36 views

Exploratory Data Analysis on banknotes data

I have a banknotes data set downloaded from here and I'm only using the first two columns: V1 and V2. I'm trying to run K-Means clustering algorithm to evaluate if the dataset is suitable for the K-...
1
vote
1answer
33 views

Clustering after PCA: Use the standardized data, or take into account the variation explained at each PC?

I am interested in clustering daily gridded data. Because of the many dimensions (gridpoints), I first perform PCA to reduce the dimensionality and keep the n-first PCs that account for at least 85% ...
1
vote
0answers
22 views

End to end k-means clustering - python

i'd like to share with you my path in a clustering exercise (via K-means using python), in order to understand if i made some errors or if there is something more that can i do. General Overview My ...
0
votes
0answers
12 views

How do I evaluate a K-Means unsupervised anomaly detection approach?

how do I evaluate K-means clustering anomaly detection method as there is no labelled data of anomaly class. To find the cluster (K), I have used the silhouette score from Scikit learn library. Scikit ...
1
vote
1answer
30 views

Clustering with Only Categorical Features

I am trying to do clustering with a bunch (24) of categorical features. I have done some research and found a lot of people recommending something such as K-Modes. I tried running K-Modes on my data ...
2
votes
2answers
30 views

Is confusion matrix possible in one column

I am performing anomaly detection using K-Means. I am working with only one column, plotting those values and then within this column I am adding some anomalies. My question is if it is possible to ...
0
votes
1answer
29 views

Confusion Matrix and AUC in univariate Anomaly Detection

In the code I upon a csv file which only has one column. The data in there in not that important just normal numbers. ...
1
vote
1answer
53 views

How to use Cosine Distance matrix for Clustering algorithms like mean-shift, DBSCAN, and optics?

I am trying to compare different clustering algorithms for my text data. I first calculated the tf-idf matrix and used it for the cosine distance matrix (cosine similarity). Then I used this distance ...
3
votes
3answers
39 views

K-Means anomaly detection not clustering anomalies

K-means anomaly detection scatter plot The following code, takes a single column from a dataset and then adds 50 anomalies to the dataset that is quite bigger than the maximum values of the dataset. ...
2
votes
1answer
35 views

How to test/train a model for realtime data with new data points and classes in a ML pipeline

First, For a text classification problem, if I have trained the model on 2 classes and it gives good accuracy. Now, when I use the model in real-time, there is a completely new class from a totally ...
3
votes
0answers
34 views

What are the data preprocessing steps required before running K-Modes?

I have a clustering task at hand. The data that I have contains only categorical variables. So, k-modes seemed like the best option. But I am not sure what are the data pre processing steps required ...
-1
votes
1answer
25 views

How to speedup K-Means used in 'for loop'

I'm trying to solve an interesting problem. One solution which seems to work well, involves using K-Means in a 'for loop'. The dataset per loop is independent and fairly small (Minibatch not required)....
2
votes
1answer
25 views
3
votes
1answer
32 views

Why is hierarchical clustering quadratic and k-means linear?

According to the internet, k-means clustering is linear in the number of data objects i.e. O(n), where n is the number of data objects. The time complexity of most of the hierarchical clustering ...
2
votes
1answer
36 views

Outlier Detection using K-Means using one column

I have done and read a csv file and then plotted the values of a single column using K-means ...
0
votes
3answers
50 views

K-Means Clustering too crowded

I have written a simple python code that opens a csv files and then clusters the values of one column. There around 10k rows This is my code ...
1
vote
1answer
17 views

Elbow method on hundreds columns and rows

So I have these vectors called matrix_ after I applied TF-IDF (term frequency-inverse document frequency), and I also converted it to dataframe ...
-1
votes
2answers
56 views

Clustering data set with multiple dimensions

I have a data set which is similar to the following: It is recipe data along with the composition of the recipe (in %) I have 91 recipes and 40 ingredients in total. I want to be able to cluster ...
0
votes
2answers
62 views

K-Means Clustering for data points with multiple attributes

I'm very new to K-Means clustering. Every example that I have seen has a two-dimensional data set. I am working to classify recipes of varying ingredient composition into families. Each recipe is ...
-1
votes
1answer
31 views

kmean clustering

I am having a problem in Matlab. I would like to use kmeans clustering and then get the value and index of the centroid. For example, if there is an $5*5$ array, we do kmeans clustering where k=2 and ...
0
votes
0answers
35 views

ValueError while Plotting a specific column using K-Means

Following a tutorial, I was able to plot an array of data using KMeans. This is a csv file ...
2
votes
0answers
40 views

Will flattening multivariate time series data before clustering make the results meaningless?

I have a large number of financial time series that I wish to do cluster analysis on. Each time series has the same length and spans multiple years of daily data (returns, volatility, etc.). As part ...
5
votes
1answer
51 views

How do I interpret my result of clustering?

I am working on a clustering problem. I have 11 features. My complete data frame has 70-80% zeros. The data had outliers that I capped at 0.5 and 0.95 percentile. However, I tried k-means (python) on ...
3
votes
1answer
60 views

What value can I gain by doing exploratory data analysis on features (and thus data) before doing clustering?

This might not be a very good question, but I would still ask if it's beneficial to do EDA before running a clustering algorithm? I understand that EDA helps us generate good and helpful insights ...
2
votes
2answers
98 views

How to deal with with rows with zero in every feature while clustering?

I am working on a clustering problem which has 13000 observations and 15 features. Around 3000 observations in the dataset has zero in every features ( i.e all values zero in 3000 rows). I am trying ...
2
votes
2answers
43 views

Is k-means with Mahalanobis a valid option for clustering?

I want more info into if k-means with Mahalanobis distance is a mathematically/methodologically correct option for datasets with different variance clusters. The steps are: Create aggregate datasets (...
2
votes
1answer
56 views

Using TSNE to Visualize Clusters in Python

I'm using TSNE to visualize my clusters but the output seems a bit strange. There are supposed to be 3 clusters but instead, there are 4 lines. Is there something wrong with how I'm visualizing them ...
4
votes
0answers
35 views

What are practical differences between kernel k-means and spectral clustering?

I've been lately wondering about kernel k-means and spectral clustering algorithms and their differences. I know that spectral clustering is a more broad term and different settings can affect the ...
2
votes
1answer
26 views

How to retrain a K-Modes model based on daily data?

I have read that retraining a model depends highly on what you are trying to achieve. I am conscious that maybe I need to retrain my model daily and after a certain time I have to train the model ...
1
vote
1answer
39 views

Google Earth Pro Satellite image segmentation using clustering

I have downloaded a satellite image from Google Earth Pro software corresponding to a particular date for a selected area around a place. I want to specifically segment the road lanes from the image ...
2
votes
2answers
77 views

KMeans clustering for Image Data

I am trying to cluster the sample of Imagenet Dataset using K-Means clustering. In this approach, I have used the below 2 approaches to get the optimal number of clusters. Elbow method From the ...
1
vote
1answer
56 views

Clustering with 0 or Null values

I want to do some clustering for a dataset where I am looking at 10,000 peoples usage of certain electronic devices. I have 11 columns; the first column is simply a URN representing each person in the ...
0
votes
1answer
28 views

Incremental modelling of kmeans in pyspark

I have a large dataset and trained the model with kmeans for the first time. I saved the model and pipeline used . Now again I started collecting data. After sufficient data is collected using old ...
0
votes
1answer
29 views

PCA before Affinity Propagation (AP)

I have a large-ish dataset (100k samples, ~100 features), that I am trying to cluster, to an unknown number of clusters. I thought of using PCA first, to reduce dimensionality, since I understand that ...
0
votes
1answer
25 views

Unsupervised Learning::Satellite Images::Single Bands

Has anyone has success with building models using KMeans for classification? I have images that only have one band and it continues to fail. My guess is that the issue is with both size of the image ...
0
votes
1answer
17 views

Topic models for non-textual data?

I am looking to employ an unsupervised clustering on a dataset where each observation has a mix of textual and non-textual features. For each observation, I combine the features into a single vector ...
3
votes
2answers
772 views

How to evaluate the K-Modes Clusters?

K-modes algorithm is available here I want to do clustering of my binary dataset. I need to specify the number of clusters that I need as an output: ...
3
votes
2answers
72 views

KMeans clusterization on documents

Whether correct or not, I'm not able to judge being myself in the early days of the Data Science. However, I have applied a Kmeans on a corpus where some random documents (very short sentences) have ...
0
votes
2answers
35 views

How can I improve the results of my clustering

I am working on a project with the idea to cluster the sound waves of key strokes on a computer. So far what I have done was recorded about 50 keystrokes per key (only have done 1 - 10 so far), found ...
0
votes
1answer
23 views

2nd, 3rd, Nth closest guesses

I have used the KMeans algorithm to create an engine that can guess the cluster that a particular set of input data will fall into. Can I use it to guess the 2nd closest cluster, 3rd closest, and so ...
2
votes
1answer
51 views

Evaluate clustering by using decision tree unsupervised learning

I am trying to evaluate some clustering results that a company did for some data but they used an evaluation method for clustering that i have never seen before. So i would like to ask your opinion ...
1
vote
1answer
78 views

will k-means clustering converge to the same results given the same data set?

I did some study on the k-means clustering algorithm. It seems that the only non-deterministic part is the centroid - initialization. Assume I have 10k data points, and a given k. I then initialize ...
1
vote
2answers
46 views

Clustering Small Text Descriptions

Im presented with a unique text classification problem. Im given a list of descriptions each containing 3-8 words. I know that there are some descriptions that are nearly the same, but the majority ...
0
votes
1answer
27 views

Customer Segmentation: Should I use a variable, representing a product, that is unpopular in the dataset for K-Means Clustering?

I am working with a data set that, besides customer age and income, tells the balance a customer has in different type of bank accounts: Checking, Shares, Investment, Savings, Deposit, Mortgage, Loan, ...
0
votes
0answers
21 views

Clustering data using KMeans centroids of base period for pattern analysis

I have a data frame consisting of 12 months of Customer Transaction Level Data. The data is unsupervised. The data is divided into 6 sets of 2 months period each. Taking first period as the base, I am ...
0
votes
1answer
37 views

color compression using k-means algorithm

I am reading about k means algorithm at this link. At ln[22] here author mentioned that Input color space is 16 million possible colors. How author came up with 16 million number here. Kindly explain....
0
votes
2answers
39 views

customer segmentation with unbalanced data

I am trying to do a customer segmentation on my transactional data and I am struggling a little bit on the best approach. Since it is an unsupervised model I can throw it to any algorithm and get some ...
-1
votes
1answer
17 views

Efficient algorithm to find the lowest value cluster in a series of values

Let's say I have a list of numeric values that tend to be grouped into some number of clusters of values that are close to one another. I'm aware of things like k-means to group these into groups of ...

1
2 3 4 5 6