Statistical Commute Analysis in Java

Question

I have a rather large commute every day - it ranges between about an hour and about an hour and half of driving.

I have been tracking my driving times, and want to continue to do so. I am capturing the date, my time of departure, my time of arrival, the route I took (there are two or three possible ones), weather conditions (wet/dry and clear/hazy/foggy), and whether I stopped (and if so, for what reason - fuel/toilet break/food break, and for how long) for every journey to and from work.

I would like to create a system to analyse this data and suggest an optimal departure time (for the next journey) based on day of the week, weather conditions, and whether i need to stop.

Anecdotally, I can see that Tuesday mornings are worse than other mornings, the earlier I leave the more likely I am to take a toilet break or a food break, and obviously that the journey takes longer on rainy or foggy days than on clear and dry days - but I would like the system to empirically tell me that!

I assume this is a machine-learning and statistical analysis problem.

However, I have absolutely no knowledge of machine-learning, or statistical methods.

What statistical methods should I use to do this kind of analysis to the point where the data will lead to suggestions like "tomorrow is Tuesday and it is going to rain, so you must leave home between 7.50 and 8.00, and take route XYZ, to get the optimal driving time. Oh and chances are you will need a toilet break - and I have factored that in"? (assume that I manually enter tomorrow’s weather forecast - I’ll look into integrating with a weather service later)

Note that this is life-hacking for me, trying to optimise the hell out of a tedious process, and it is very personal - specific to me and my habits, specific to this route, and specific to the morning/evening commute times. Google Maps with Traffic, TomTom with IQ, and Waze do very well in the more open-ended situations of ad-hoc driving-time prediction. Even Apple is happy to tell me on my iPhone notification screen how long it will take me to get home if I leave right now.

Also note, it appears to me that traffic is not a consideration - that is to say, I do not think I need to know the actual traffic conditions - traffic is a function of day of the week and weather. For example, there are more people on the roads on Monday and Tuesday mornings, and people drive more slowly, and more people are in cars (opting to drive instead of cycle or take public transport) when it rains.

To what extent can I let the data do all the talking? I have a somewhat ambiguous hidden agenda which may not be apparent from the data;

• I should be at work at 9.30 (i.,e. 9.15 +/- 15 minutes) every day, but the occasional 10am arrival is OK
• I want to leave home as late as possible, and yet arrive at work as early as possible
• I want to leave work as early as possible, and yet have done at least 8 hours’ work
• it is OK for me to, say, leave half an hour early on one day but stay late on another to compensate

I think I can come up with a procedural formula that can encompass all of these rules, but my gut feeling is that statistical analysis can make it a lot smarter.

Apart from the methods of analysis, the technology stack is not an issue. Java is my language of choice - I am quite familiar with programming in it, and in creating web applications.

Assuming that it is possible, are there Java libraries that can provide the requisite methods?

What limitations are there? I want to keep capturing more and more data every day, making the data set bigger, hopefully, making the prediction more accurate.

What other ways are there to do it? Can I push this data into, say, Wolfram Programming Cloud, or maybe something Google provides to get the desired results?

Answer 1

You do want to model the traffic, at least over a work day, otherwise it wouldn't matter what time you traveled! Absent any data, I'd assume there isn't much variance over the working week, but that's one thing the data will quickly confirm or refute. If it is varying, you can use a different model for each day.

You have two variables; the departure times from home and work, respectively. Let's call them t_h and t_w. Let's call the commute time T_c(t), where t is the time of day. You can estimate this function from the data, so I'll assume it is given.

You want to maximize c t_h - (1-c) t_w subject to the constraints t_h + T_c(t_h) < 9.5 and t_w > t_h + T_c(t_h) + 8

where c is a constant you can set to adjust the relative importance of leaving home early relative to leaving work early. You should be able to solve this numerical optimization problem with Mathematica, MATLAB, or something similar. I would not recommend Java; it's not meant for this. The only tricky part is estimating T_c. You know that it's a non-negative function, so you could use the standard trick of estimating it's logarithm (say, with kernels) and exponentiating. For implementation with Mathematica see Smoothing Data, Filling Missing Data, and Nonparametric Fitting and Constrained Optimization.

Answer 2

Basically your machine-learning problem is: given [day-of-week, weather, departure-time, route], predict arrival time, then obtain travel time. Once your model is solid, you can then predict your travel time for each potential departure time and route, and choose the lowest.

You don't really need to factor in the pit stops, if you think they're a function of the rest, or a random variable altogether.

If you want a tool that can do the machine-learning aspect 'out of the box' you should try Weka. You'll have to encode the data in the specific format it expects, but other than that you won't have to do any coding (i.e. you won't need to code any of the actual machine-learning algorithms).

I would discretize your departure time to make sure you have enough data. Weka will let you try out the different algorithms, see which one is best. Note that it's a regression problem as opposed to a classification problem, so only a subset of the algorithms apply. Once you've got the Weka part figured out, you can also call it programmatically, which will allow you to code the interface you want, and potentially include the automatic weather retrieval.