I would like to make predictions about how crowded a location (postal code mapped to lat/lng coords) would be. I would like to provide predictions to questions like: what’s the crowd density going to be like for the area surrounding postal code 12345 for 50metre next Monday between 6pm to 8pm?

For a start I’m working with historical transaction data (as a proxy for human activity) which contain time stamps and postal codes. I would also like to incorporate new transaction data into my model as they occur.

Does this make sense?

  1. Filter the historical transactions based on requested time (between 6pm to 8pm).
  2. Cluster the transactions based on the desired proximity to get a daily transaction totals for the area.
  3. Use the daily totals as training data for my regression model.
  4. Use the model to predict

Would I need to rerun the entire process to account for new data which arrives in real time (e.g every 5mins)? Are there any incremental ML models which could be suitable?

  • $\begingroup$ How much ahead in time are you predicting? Is it a week ahead, a few days, or a few hours? What new information will you get in 5 minutes that could change your predictions? With a larger lookahead, re-training the data once everyday should suffice. If the lookahead is a few hours, i.e. at 4PM you want to predict the density at 6PM, then it is likely that adding 5 minutes of the latest data could indicate a change in density, eg. more than usual density at 4PM could indicate an irregular event happening at the location, which could mean higher density than usual at 6PM as well. $\endgroup$ – raghu May 12 '18 at 7:56

Are there any incremental ML models which could be suitable?

You have to be able to access the model when it's in production mode and actually be able to update it with a new version. all ML models should be adjusted when new data comes because of the so-called data shifting. you could periodically evaluate how much the model is performing.

I suggest modeling the problem as a sequence model, use multiple output at each day to predict densities for each postal code.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.