# Are there any machine learning techniques to identify points on plots/ images?

I have data for each vehicle's lateral position over time and lane number as shown in these 3 plots in the image and sample data below.

> a
Frame.ID   xcoord Lane
1       452 27.39400    3
2       453 27.38331    3
3       454 27.42999    3
4       455 27.46512    3
5       456 27.49066    3


The lateral position varies over time because a human driver does not have perfect control over vehicle's position. The lane change maneuver starts when the lateral position changes drastically and ends when the variation becomes 'normal' again. This can not be identified from the data directly. I have to manually look at each vehicle's plot to determine the start and end points of lane change maneuver in order to estimate the duration of lane change. But I have thousands of vehicles in the data set. Could you please direct me to any relevant image analysis/ machine learning algorithm that could be trained to identify these points? I work in R. Thanks in advance.

• Have you tried to identify mathematically what it is you do when you manually classify the lane change manoeuvres? Are you broadly looking for a change from a period of stable close to zero gradient of the lane position function followed by a large increase in the magnitude of gradient, leading to either another period of close to zero gradient or the end of the data ? – image_doctor Sep 6 '15 at 9:08
• Do you have a number of the original images for us to experiment with ? – image_doctor Sep 6 '15 at 9:26
• The axis and scales are legends not particularly consistent across the example images, is there a way to standardise the plots, or do you have no control over the image creation ? – image_doctor Sep 6 '15 at 13:06
• Yes, I understand that you want to identify the end of the change lane maneuver, but if you already have the lane of the vehicle at each time, then it is not hard to detect those changes. I would start by defining when we should consider that the vehicle is not changing lanes anymore (e.g. after a given number of seconds driving on the same lane). You could use a window to detect segments in which the vehicle keeps the same lane and the points at the start and end of such segments describe your "start of lane change" and "end of lane change", respectively. – Robert Smith Sep 19 '15 at 21:35
• Great. I thought you didn't have origin and target lane but If you always have them, your solution should work and additionally uses the data you already have to construct a definition of lane change. – Robert Smith Sep 20 '15 at 17:07

A first derivative, on the surface, would do it. However, the data you show have a great deal of noise in them, so we need a way to evaluate the first derivative in a somewhat noiseless way, or at least within a frequency domain that eliminates the jitter and preserves the major derivative change.

Wavelet analysis could achieve this for you, especially if you use the first derivative of a Gaussian as your mother wavelet. R has some decent wavelet packages (see r-project.org for starters). If you do the wavelet transform at short scales, this will identify the locations of the bits of jitter in the steering. If you do it at larger scales (i.e. lower frequency), you can likely find just the lane shifts and not the little jitters.

If you train the transform up with a reasonable data set, you should be able to identify a scale or range of scales that corresponds to lane changes. But note that if you don't figure that out, this is something like O(n^2), so try to narrow the scale range down a bit to save compute time.

Looks like you could just look for a few seconds of higher then noise derivative. Just calculate the absolute value of the finite difference from each timestep to the last ( or one of the former ) and wait for a series of high values. That's when a lane change occurs.

• This is actually what I did in the first place. The problem is that the difference threshold and "high" values are very subjective because each vehicle's journey is different. – umair durrani Sep 16 '15 at 1:02

Try the changepoint package. I used it in a similar case.

Changepoint analysis is the statistical name for methods that detect changes between two "regimes". A car staying in a lane is a line with gradient 0 at the midpoint of a lane. You can easily fit a statistical model to cars driving in lanes. A car changing lane is driving along a line with a gradient that isn't 0. The model has changed. Changepoint analysis, and the changepoint package, is exactly what you need to determine the point when a model changes from y=a' (straight and level) toy=a+bx` (going up or down).

• This is essentially a link-only answer and tends to be discouraged on SE. Maybe you can elaborate what it is and why it's helpful. – Sean Owen Sep 15 '15 at 15:14
• @AlbertoD The archaic language of the vignette you shared is not helpful for someone new to the concept of changepoint analysis. – umair durrani Sep 16 '15 at 1:00
• @AlbertoD Could you please provide any example of how you used the cp package in your case? – umair durrani Sep 22 '15 at 0:55