1
$\begingroup$

I have developed an iterative process via which I can collect data in batches. The data are points in a predefined 3D space. I am trying to explore and locate clusters in that 3D space based on my data. After some batches have been collected I can locate and create the clusters. As I collect even more data though, small refinements are being made and the new results are not really worth the effort. How can determine whether a new batch is "worth" collecting? Is there some metric I can use to measure how much the batches I have are "similar" or whether a new random batch will affect the overall process?

$\endgroup$
  • $\begingroup$ What are the results / objective? $\endgroup$ – user2974951 Dec 5 '18 at 12:43
  • $\begingroup$ The objective is to roughly describe the clusters in the space. I am interested in the number and the approximate space they cover. $\endgroup$ – Nikolaos Paschos Dec 5 '18 at 12:45
  • $\begingroup$ If this is an unsupervised analysis (that is you do not know the actual labels of your data) then you can use metrics used in clustering, such as Silhouette coefficient. Have a look at the section about evaluation en.wikipedia.org/wiki/Cluster_analysis $\endgroup$ – user2974951 Dec 5 '18 at 12:52
  • $\begingroup$ How big is the difference in the clustering before and after the last batch? How much money does it bring to your project? How much money does the new data cost? Time * time cost $\endgroup$ – keiv.fly Dec 5 '18 at 23:40
  • $\begingroup$ Sorry for the late response, I wanted to thoroughly check what you suggested @user2974951. I tried calculating various measures such as Silhouette and I noticed that it actually depends. To begin with, Silhouette is a measure describing each individual cluster. Hence I tried using either the one with more noticeable difference or the avg difference. Both approaches yielded interesting results. Now I guess I need to figure out a threshold that I consider a "healthy" Silhouette and stop iterating at that point. $\endgroup$ – Nikolaos Paschos Dec 7 '18 at 9:35
1
$\begingroup$

Quantify how much your clusters change with each batch.

Then stop if the change becomes smaller than a threshold.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I see what you're saying but I fail to understand this: change is measured between two states, state A (previous) and B (current). This change depends on the data that triggered it. Each of my batches contain data that describe part of the clusters that I have already discovered, not necessarily all of the clusters either. Hence the change can vary and strongly depends on the new batch. Is there a "template" batch I can use in order to measure the change? $\endgroup$ – Nikolaos Paschos Dec 10 '18 at 8:09
  • $\begingroup$ All of the data prior to the current batch? $\endgroup$ – Has QUIT--Anony-Mousse Dec 11 '18 at 6:38
  • $\begingroup$ I get that, but the results will vary depending on the new batch. Different new batches will yield different results with the same previously collected data. Is that not an issue? $\endgroup$ – Nikolaos Paschos Dec 11 '18 at 14:47
  • $\begingroup$ Then you probably never have "enough" data, if your data and hence clusters keeps on changing. $\endgroup$ – Has QUIT--Anony-Mousse Dec 11 '18 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.