3
$\begingroup$

My data is a group of 10 thousand points (each having an node location (x,y)) that are spread across a plane. They are also chromatically-colored based on their weight.

I need to finalize a bayesian nonparametric clustering method that groups points on mainly weight, but also on distance: that is, clusters are by defintion have some relevance to distance, but there are clear topological distinguishing factors between the first quarter and the last quarter of data (I say quarter as an arbitrary amount; in reality, the exact number and topology of the cluster changes through iterations).

As you can see in this picture, I’ve tried to use notability to create a crude chromatically-colored image of data with varying cluster topology types; over each iteration of my algorithm, the clusters, as mentioned, change location (based on their weight) and their shape

As you can see in this picture above, I’ve tried to use notability to create a crude chromatically-colored image of data with varying cluster topology types; over each iteration of my algorithm, the clusters, as mentioned, change location (based on their weight) and their shape, and some overlap (and the possiblility of new clusters (or a possible decrease in the total number) is very high per iteration, where this image represents one iteration of x points)

Additionally, since I am doing this analysis with data via python, i was thinking of using the T-SNE machine learning package as a substitute for a generic clustering method, but I only have a limited knowledge on its functionality. Also, since my data is based on the same weighted scale, it may be overkill.

EDIT: I changed the picture to show overlapping clusters, so it is clearer what I mean. However, keep in mind even these visible clusters are not homogenous in weight (they still vary but in a small threshold). Sure there is noise, but I really want to treat each cluster independently to see each cluster’s behavior over time (as well as clusters that are newly formed, hence the nonparametric method)

$\endgroup$
14
  • 2
    $\begingroup$ Did you try DBScan? $\endgroup$ – Itamar Mushkin Jul 13 '20 at 11:09
  • $\begingroup$ What did you try? $\endgroup$ – Itamar Mushkin Jul 13 '20 at 11:10
  • $\begingroup$ @ItamarMushkin I did try dbscan alone but the very big issue is that dbscan doesnt understand classifications by weight. Weight does not directly correlate with importance though: that is, I need to cluster high weights together and low weights together (and both are equally important), but almost always different clusters with different topologies overlap into one big cluster (So DBSCAN puts them together). Because of this, i have tried doing manually separating the values by weight and THEN doing DBSCAN but this is an extremely tedious for 1000s of iterations. What do you recommend I do? $\endgroup$ – ChessGrandMaster Jul 13 '20 at 17:31
  • $\begingroup$ @ItamarMushkin It is because of this that I might have to resort to some unecessarily complicated algorithm like tsne for relatively basic and not heavily varied data (after all, the only variation is based on weight and their distance to one another), which i really dont want to do. $\endgroup$ – ChessGrandMaster Jul 13 '20 at 17:34
  • $\begingroup$ @ItamarMushkin To make what im saying clearer, i updated the image as well $\endgroup$ – ChessGrandMaster Jul 13 '20 at 17:42
0
$\begingroup$

One option is spectral clustering. Spectral clustering can find the "connectedness" in data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.