How to predict an approximate weekly/monthly number, when the Unique Daily Visitors for that week/month are already known

Question

I am trying to come up with a formula or machine learning algorithm using which I can approximately predict the weekly or monthly users.

What to keep in mind is that I already have counts for the unique visitors per day for the week/months that I would like to make a near accurate prediction. Here, simply summing the daily unique users would not work, as they can be unique on one day but not on two days as they can have a session lasting over 2 days.

This method is to serve as an alternative to running a Spark job on the whole week/month data in order to save time and resources - Is this possible?

I have looked at Time Series and Linear Regression, but need more clarification on the possible approaches and also on any work-arounds?

Answer 1

To calculate the (estimated) value of the monthly/weekly unique visitors based on the per day count you may use the HyperLogLog algorihm.

This will give you the historical values and you can use any time series approach to predict future counts

Answer 2

You can't retrospectively do it just with counts of unique visitors per day. If you represent the unique users on each day by sets $A_1, A_2, \dots, A_n$, the union can be as small as $|A_1|$, if all sets are equal, or as large as $|A_1| + \dots + |A_n|$ if all sets a pairwise disjoint.

If you could estimate average number of days r per month a user visited the site (over the users that visited at least once), then there were exactly $(|A_1| + \dots + |A_n|)/r$ unique visitors that month.

A nice proper way to do this, starting with new online data, is to use Hyperloglog or other memory-efficient approximate counting algorithms, as Marmite Bomber suggests.

Answer 3

I do not know what industry you are working in but predicting unique users can be difficult at times because of factors outside of your control. For example if your company tries a new advertising campaign that targets a new segment it is hard to calculate a range because they are a new eco system interacting with your previously defined set of rules.