I have a dataset that contains several measures from various widgets on a daily basis. While the widgets remain relatively stable over time, sometimes there are legitimate reasons for one to disappear and another to appear in the data as a whole. Occasionally, a widget will just disappear and so the dataset is incomplete, invalidating the whole dataset for that day.

What I am looking for is a method of comparing the current set of widgets with another set of widgets to detect if any widgets are missing. I am not trying to create the values, just identify that they are missing. I could do time-series, but that feels like overkill on so many widgets and there are multiple attributes on which data might be missing. I was hoping for something more set based that might account for the regular changes in widgets but detecting the unusual dropouts. I am sure I just need to adjust the way I am thinking about the problem.

Any ideas would be much appreciated?

  • $\begingroup$ concept is good and any sample data? $\endgroup$ – sai saran Nov 15 '18 at 3:49
  • $\begingroup$ Unfortunately, it is proprietary data, but I'll try to document a proxy of the data. $\endgroup$ – Skiddles Nov 15 '18 at 14:29
  • $\begingroup$ Something I don't get: what happens to your data when a widget "disappears"? $\endgroup$ – anymous.asker Nov 17 '18 at 18:47
  • $\begingroup$ When the widgets disappear, part of the whole is missing. $\endgroup$ – Skiddles Nov 18 '18 at 1:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.