I have developed an iterative process via which I can collect data in batches. The data are points in a predefined 3D space. I am trying to explore and locate clusters in that 3D space based on my data. After some batches have been collected I can locate and create the clusters. As I collect even more data though, small refinements are being made and the new results are not really worth the effort. How can determine whether a new batch is "worth" collecting? Is there some metric I can use to measure how much the batches I have are "similar" or whether a new random batch will affect the overall process?

  • $\begingroup$ What are the results / objective? $\endgroup$ Commented Dec 5, 2018 at 12:43
  • $\begingroup$ The objective is to roughly describe the clusters in the space. I am interested in the number and the approximate space they cover. $\endgroup$ Commented Dec 5, 2018 at 12:45
  • $\begingroup$ If this is an unsupervised analysis (that is you do not know the actual labels of your data) then you can use metrics used in clustering, such as Silhouette coefficient. Have a look at the section about evaluation en.wikipedia.org/wiki/Cluster_analysis $\endgroup$ Commented Dec 5, 2018 at 12:52
  • $\begingroup$ How big is the difference in the clustering before and after the last batch? How much money does it bring to your project? How much money does the new data cost? Time * time cost $\endgroup$
    – keiv.fly
    Commented Dec 5, 2018 at 23:40
  • $\begingroup$ Sorry for the late response, I wanted to thoroughly check what you suggested @user2974951. I tried calculating various measures such as Silhouette and I noticed that it actually depends. To begin with, Silhouette is a measure describing each individual cluster. Hence I tried using either the one with more noticeable difference or the avg difference. Both approaches yielded interesting results. Now I guess I need to figure out a threshold that I consider a "healthy" Silhouette and stop iterating at that point. $\endgroup$ Commented Dec 7, 2018 at 9:35

1 Answer 1


Quantify how much your clusters change with each batch.

Then stop if the change becomes smaller than a threshold.

  • $\begingroup$ I see what you're saying but I fail to understand this: change is measured between two states, state A (previous) and B (current). This change depends on the data that triggered it. Each of my batches contain data that describe part of the clusters that I have already discovered, not necessarily all of the clusters either. Hence the change can vary and strongly depends on the new batch. Is there a "template" batch I can use in order to measure the change? $\endgroup$ Commented Dec 10, 2018 at 8:09
  • $\begingroup$ All of the data prior to the current batch? $\endgroup$ Commented Dec 11, 2018 at 6:38
  • $\begingroup$ I get that, but the results will vary depending on the new batch. Different new batches will yield different results with the same previously collected data. Is that not an issue? $\endgroup$ Commented Dec 11, 2018 at 14:47
  • $\begingroup$ Then you probably never have "enough" data, if your data and hence clusters keeps on changing. $\endgroup$ Commented Dec 11, 2018 at 19:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.