Hot answers tagged

4

If you want to test whether your algorithm works as expected, I'd use sklearn datasets. They allow you to create simple synthetic 2D data with certain properties: circles, half moons, etc. If you want "real" datasets, here is an interesting resource found after a brief search: https://uni.hi.is/helmut/2019/06/20/datasets-for-dbscan-evaluation/ It seems to ...


2

Dont try to visually confirm it. You are plotting your clustering resutls in ONLY two dimensions and you expect that all of the information is in these two dimesnions. That is very unlikely. If you plot 3 dimensions you will see even more seperability and it will make a bit more sense. In any case you need a metric for example Silhouette that tells you how ...


2

Each of those selected clustering algorithms can be fit using cosine distances in scikit-learn: from sklearn.cluster import DBSCAN, MeanShift, OPTICS from sklearn.metrics.pairwise import cosine_distances # Define clustering algorithms algorithms = [DBSCAN, MeanShift, OPTICS] # Placeholder for results results = dict.fromkeys((a.__name__ for a in algorithms)...


1

Clustering in X,Y,Value Clustering in 3d is great. But be careful with feature scaling in this case. Presumably X, Y have the same scales - so unless you want to treat the different directions differently, make sure not not apply any normalization. As that would distort the grid. Your Value column on the order hand, might be on a very different scale from ...


1

First of all K-means is a partitioning algorithm where as DBSCAN is a Density clustering algorithm. K-means tries to find cluster centers that are representative of certain regions of the data. DBSCAN doesn’t require every point be assigned to a cluster and hence doesn’t partition the data, but instead extracts the dense clusters and leaves sparse background ...


1

Kaggle has some nice datasets available, including the classic Iris dataset. Take a look and pick one that looks interesting. There are some impactful real-world data sets there, including COVID-19 related data sets. Something on the lighter side might be this scrubbed Iris data set posted not long ago. EDIT: to elaborate on COVID-19, Kaggle has the COVID-...


1

Don't try to find parameters by brute force. Instead, analyze your data. The choice of minpts is application driven - how noisy your data is, how many points you require for a point to be considered important. Based on this, you can choose epsilon based on the k-distance plot. Try projecting your data into different views when you have multiple dimensions. ...


1

It sounds like you've got the right idea. As you say, lemmatize those descriptions, then vectorize using (word count, TF-IDF, or binary encoding). Try passing that to DBscan, and use the clusters it returns as the groupings for your descriptions. While you're using DBscan, make sure to try varying the metric you use, as quite a few are available. Finding ...


1

On the top of my head: with regular clustering techniques you could try to use text-specific distance/similarity measures instead of only considering distinct words as elements. There are hybrid string similarity measures such as SoftTFIDF which take into account character-based and word-based similarity. use lemmas instead of words in order to facilitate ...


1

Start with what the parameters mean. $\varepsilon$ is the search radius around each point. You need this search radius to be small enough that it can't fully "bridge the gap" between the clusters. If the gap width is variable, $\varepsilon$ needs to be small enough to accommodate the narrowest gap. Note that we can exclude the occasional straying point from ...


Only top voted, non community-wiki answers of a minimum length are eligible