Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

If I have a retail store and have a way to measure how many people enter my store every minute, and timestamp that data, how can I predict future foot traffic?

I have looked into machine learning algorithms, but I'm not sure which one to use. In my test data, a year over year trend is more accurate compared to other things I've tried, like KNN (with what I think are sensible parameters and distance function).

It almost seems like this could be similar to financial modeling, where you deal with time series data. Any ideas?

share|improve this question

The problem with models like KNN is that they do not take into account seasonality (time-dependent variations in trend). To take those into account, you should use Time Series analysis.

For count data, such as yours, you can use generalized linear auto-regressive moving average models (GLARMA). Fortunately, there is an R package that implements them (glarma).

The vignette is a good resource for the theory behind the tool.

share|improve this answer
Another very useful package for forecasting and time series analysis is forecast by Prof. Rob J. Hyndman. – Def_Os Jun 16 '14 at 18:05
Do you know if this is already implemented in any other language? I'm not exactly a pro with R. I will definitely read the paper at least. – user1132959 Jun 17 '14 at 1:18
I am not familiar with one. If you would like to use python, you can use the rpy2 package to call the glarma function while doing most of the rest of the programming in python. Most other languages have such a connector as well. – Christopher Louden Jun 17 '14 at 12:59
I will look into this option. Thanks. – user1132959 Jun 23 '14 at 12:59

I think Christopher's answers above are entirely sensible. As an alternate approach (or perhaps just in addition to the advise he's given), I might start by just visualizing the data a bit to try get a rough sense of what's going on.

If you haven't already done this, you might try adding a date's month and day of week as features -- if you end up sticking with KNN, this will help the model pick up seasonality.

As a different way of taking this on, you might consider starting with a really, really basic model (like OLS).. these often go a long way in generating reasonable predictions.

Finally, the more we know about your data, the easier it will be for us to help generate suggestions -- What time frame are you observing? What are the features you're currently using? etc.

Hope this helps --

share|improve this answer
Yes, visualization is an essential first step in any analysis. – Christopher Louden Jun 16 '14 at 18:42
I did add the month, day of month, day of week, and year as features. I even tried a linearly decreasing "Recentness" value. I don't think I have tried OLS. I'm observing a time frame that could range anywhere from a couple weeks to multiple years. As far as visualizing it goes, I did try to do that. The problem is, we want the software to be able to predict automatically, without human intervention, for different customers. – user1132959 Jun 17 '14 at 1:24

You could try Neural Network. You can find 2 great explanations on how to apply NN on time series here and here.

Note that it is best practice to :

  • Deseasonalize/detrend the input data (so that the NN will not learn the seasonality).
  • Rescale/Normalize the input data.

Because what you are looking for is a regression problem, the activation functions should be linear and not sigmoid or tanh and you aim to minimize the sum-of-squares error (as opposition to the maximization of the negative log-likelihood in a classification problem).

share|improve this answer
I was looking into Neural Networks as an option, but didn't know what kind of parameters I would use. I'll have to give those a shot. – user1132959 Jun 23 '14 at 12:54
Sigmoid and Tanh are fine, nonlinearity is wanted to learn more complex interactions and the weights will map these to any range the network deems necessary – Jan van der Vegt Jul 20 at 11:35

As @Christopher Lauden mentioned above, time-series analysis is most appropriate for this sort of thing. If, however, you wished to do a more traditional "machine learning approach", something that I have done in the past is to block up your data into overlapping windows of time as features, then use it to predict the next days (or weeks) traffic.

Your feature matrix would be something like:

t1 | t2 | ... | tN
t2 | t3 | ... | tN+1
t3 | t4 | ... | tN+2
tW | tW+1 | ... |tN+W

where tI is the traffic on day I. The feature you'll be predicting is the traffic on the day after the last column. In essence, use a window of traffic to predict the next day's traffic.

Any sort of ML model would work for this.


In response to the question, "can you elaborate on how you use this feature matrix":

The feature matrix has values indicating past traffic over a period of time (for instance, hourly traffic over 1 week), and we use this to predict traffic for some specified time period in the future. We take our historic data and build a feature matrix of historic traffic and label this with the traffic at some period in the future (e.g. 2 days after the window in the feature). Using some sort of regression machine learning model, we can take historic traffic data, and try and build a model that can predict how traffic moved in our historic data set. The presumption is that future traffic will resemble past traffic.

share|improve this answer
Could you elaborate on how you would use this feature matrix? Are you trying to learn based on change of traffic across days? – user1132959 Jun 18 '14 at 19:19
I edited the response to hopefully give more clarity. – gallamine Jun 19 '14 at 14:32

Well, firstly, I would not even use things like Machine learning without having in depth knowledge. Simplistic things I would do if I had this time series is:

  1. Write sql queries to understand which of the times you have the busiest, average and low foot traffic.
  2. Then try to visualize the whole time series, and you could use basic pattern matching algorithms to pick up patterns.

This two things will help you understand what your data set is telling you. Then, with that in hand, you will probably be in a better state to use machine learning algorithms.

Also, I'm currently working in building something on time series, and using time series analysis will help you much more than machine learning. For example, there are pattern recognition algorithms that you can use that uses every day data to show patterns, and ones which use up to as much as 3 to 6 months of data to catch a pattern.

share|improve this answer

I would advice against using a neural network or equivalent as, I assume, you have got such a good prior based on your experience with the store (ie that there are probably day-to-day / seasonal trends and some level of smoothness) and I imagine a relatively small amount of data. A better option IMO would be to go for a kernel method such as a Gaussian Process or SVM.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.