Hot answers tagged

9 votes
Accepted

Knn distance plot for determining eps of DBSCAN

You take the last column of that matrix sort descending plot index, distance hope to see a knee (if the distance does not work well. there might be none)
Has QUIT--Anony-Mousse's user avatar
7 votes

KMeans vs. DBSCAN

In short, KMeans is a distance based clustering technique where depending on the distance between the data points your initialization(usually kmeans++) and clustering works. In kmeans, you initialize ...
karthikeyan mg's user avatar
4 votes
Accepted

Python clustering and labels

As the algorithm should not change the order of the lists you could just add the clusters list cities["cluster"] = cluster If you are really paranoid you can ...
El Burro's user avatar
  • 790
4 votes

What is slowing down classic DBSCAN algorithm

Write the DBSCAN code yourself. Run the code. Observe that your code likely will be a lot slower than the libraries code.
Has QUIT--Anony-Mousse's user avatar
4 votes

Can you l2 normalize word2vec vectors for density clustering?

Euclidean distances between l2-normalized word vectors is equivalent to the angular distance between the word vectors. This is not the same as cosine distance, but it is very similar. It is worth ...
Leland McInnes's user avatar
4 votes
Accepted

Nice real data sets for testing DBSCAN?

If you want to test whether your algorithm works as expected, I'd use sklearn datasets. They allow you to create simple synthetic 2D data with certain properties: circles, half moons, etc. If you ...
Valentin Calomme's user avatar
3 votes

Can you l2 normalize word2vec vectors for density clustering?

You can do DBSCAN, OPTICS, HDBSCAN with cosine similarity - they do not expect or require Euclidean distances. Yes, one version of cosine distance corresponds nicely to Euclidean distance of L2 ...
Has QUIT--Anony-Mousse's user avatar
3 votes

How to use Cosine Distance matrix for Clustering algorithms like mean-shift, DBSCAN, and optics?

Several scikit-learn clustering algorithms can be fit using cosine distances: ...
Brian Spiering's user avatar
2 votes

How to use precomputed distance matrix and min_sample for DBSCAN clustering method?

DBSCAN does not guarantee a minimum cluster size. There are known situations, c.f. Wikipedia, where a cluster can have fewer than "minPts" points. Furthermore, it has the concept of noise: points that ...
Has QUIT--Anony-Mousse's user avatar
2 votes

Clustering documents - how to evaluate results?

None of the internal evaluation metrics will work well on text in my experience. Probably because of the curse of dimensionality. Furthermore, DBSCAN does not cluster everything, but can also produce ...
Has QUIT--Anony-Mousse's user avatar
2 votes
Accepted

Clustering of variants of similar news articles

You don't want clustering. What you are looking for is near duplicate detection. Use minhash. Apparently that is what Google News uses for exactly this purpose.
Has QUIT--Anony-Mousse's user avatar
2 votes

KMeans vs. DBSCAN

The main difference is that they work completely differently and solve different problems. Kmeans is a least-squares optimization, whereas DBSCAN finds density-connected regions. Which technique is ...
Has QUIT--Anony-Mousse's user avatar
2 votes

How do we interpret the outputs of DBSCAN clustering?

Don't blindly copy code from the internet. You copied code that scales data. Hence, the axis changed. But that wasn't DBSCAN, but that was you invoking StandardScaler. It is important that you ...
Has QUIT--Anony-Mousse's user avatar
2 votes
Accepted

Is there an oriented clustering algorithm?

If I remember correctly, non-negative matrix factorization (NMF) can be used as a clustering approach that can recover clusters that are along vectors, for example. It may work for your dataset. It ...
Matthew's user avatar
  • 1,284
2 votes
Accepted

DBSCAN - Space complexity of O(n)?

You can run DBSCAN without storing the distances in a matrix. This has the drawback that each time you visit a point, you have to recalculate all the relevant distances, which requires more time. ...
Sotiris's user avatar
  • 61
2 votes
Accepted

Preparing dataframe to carry k-means clustering

No, it does not make sense to encore the data this way. You use Euclidean distance. You need to encode the variables in a way that Euclidean distance computes a similarity that is useful for your ...
Has QUIT--Anony-Mousse's user avatar
2 votes
Accepted

Are DBSCAN and dbscan from the sklearn.cluster package different?

Such questions are easily answered if you check the source code yourself. https://github.com/scikit-learn/scikit-learn/blob/1495f69242646d239d89a5713982946b8ffcf9d9/sklearn/cluster/dbscan_.py#L350 ...
Has QUIT--Anony-Mousse's user avatar
2 votes

Plots of data using DBSCAN algorithm not making sense

Dont try to visually confirm it. You are plotting your clustering resutls in ONLY two dimensions and you expect that all of the information is in these two dimesnions. That is very unlikely. If you ...
Noah Weber's user avatar
  • 5,669
2 votes

I need help with which features to use for clustering

PCA is interesting but hard to interpret because it is a linear algorithm. Consequently, the result using many features will be probably not OK, overall if you have non-linear correlations or complex ...
Nicolas Martin's user avatar
2 votes
Accepted

In DBSCAN, can the distance between a Noise Point and Border Point be less than Epsilon?

Yes you are correct. I just gave an example check this image
Banarasi Vaibhav's user avatar
2 votes

Why is UMAP used in combination with other Clustering Algorithm?

UMAP is not for clustering. It is just for dimensionality reduction. In the link you provided, UMAP is not used for clustering, just for dimensionality reduction. Specifically, they use BERTopic, ...
noe's user avatar
  • 25.6k
1 vote
Accepted

what arguments should I pass to dbscan or optic in order to divide the data in a specific way

Start with what the parameters mean. $\varepsilon$ is the search radius around each point. You need this search radius to be small enough that it can't fully "bridge the gap" between the clusters. If ...
Reinstate Monica's user avatar
1 vote
Accepted

T-DBSCAN - Implementing STOP logic

Just to point out a minor confusion that there seems to be in the wording: there is mixed use of the the words temporarily and temporally. [OP has since corrected this] We really only care about the ...
n1k31t4's user avatar
  • 14.8k
1 vote

How to properly use approximate_predict() with HDBSCAN clusterer for text clustering (NLP)?

Why do you recompute the distance matrix? Just compute the 1x5000 vector directly for all new points. You can even do this in batches, then feed one row at a time to the predictor.
Has QUIT--Anony-Mousse's user avatar
1 vote

DBSCAN clustering on document [updated]?

Bilgin! Anony-Mousse puts right questions and gives good suggestions. Before you use the self-implemented DBSCAN code - write it on paper. Perhaps it is not the best algorithm at all for your ...
zina's user avatar
  • 72
1 vote

Clustering based on geolocation pair

Define your own distance function. I suggest you simply use dist(x,y)=haversine(x[0],y[0])+haversine(x[1],y[1])
Has QUIT--Anony-Mousse's user avatar
1 vote

Is there any clustering algorithm to find longest continuous subsequences?

What might help is a custom distance computation as input to the clustering algorithm. These algorithms usually take Euclidean distance as a measure of dissimilarity. You can try DBSCAN (in Python ...
raghu's user avatar
  • 641
1 vote

What is slowing down classic DBSCAN algorithm

As @Anony-Mousse pointed it, on DBSCAN index structures are often used in order to decrease execution times. K-d-trees are one example but this one works well just in small dimensions. You had right ...
KyBe's user avatar
  • 410
1 vote

How do we interpret the outputs of DBSCAN clustering?

The output from db_scan.labels_ is the assigned cluster value for each of the points that you provide as input to the algorithm. You provided 20 points, so there ...
n1k31t4's user avatar
  • 14.8k
1 vote

How do I right feature selection for DBSCAN?

Automatic weighting will likely not be enough. For examples standard scaler will assign twice as much weight to the one-hot encoded parts than to the other attributes. Plus, it is based on the ...
Has QUIT--Anony-Mousse's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible