3
$\begingroup$

I'm working on the commaai speedchallenge. The goal of the challenge is to predict the speed of a car based on a dashcam video. So far all the examples that I found (example 1, example 2) use some kind of method that takes in the information at frames $x_{t-h}, \ldots, x_{t-2}, x_{t-1}$ when trying to predict the speed at frame $x_t$. I think this makes sense, because you can get an idea of the speed by looking at differences between two frames. Without a reference point (with just one frame) it would be very hard to predict the car speed..

Well apparently not! While looking for solutions I also found this repository. I'll summarise the approach that is used:

  • Rescale the frames to 50x50 images, and divide them into two sets: X_train and X_test.
  • Take X_train, X_test and pass them through the first two blocks of a convolutional network called VGG16 that is already trained, so that you get output X_train_features and X_test_features. The writer of the code explains that he wants to use transfer learning because his computer specs are not good enough to train a network himself.
  • Train a linear regressor based on the extracted features and compute the mean squared error.

If I understand the idea correctly, the author of this code just looks at each frame individually, passes the frame through the first two convolutional blocks of the network VGG16, and predicts the speed of the car based on the extracted features, without looking at earlier frames!

I've tested the code and it works, it has a mean square error just above 1!

Question: Am I missing something, or can you predict the speed of a car just by looking at the features of a single frame? If the answer is yes; then how?

I know that you can take the weights from the early layers of trained convolutional networks and use them for other purposes. This is possible because the early layers of convolutional networks usually just filter for very general/coarse shapes. What I just don't understand is how you could predict something that is relative to other frames based on just those low level features. If a human were to look at a single frame in the data, he would not be able to predict the speed of the car correctly, right?

$\endgroup$
2
$\begingroup$

There are some visual indicators of speed that should be present even in one frame. An easy example to think of is "blur", looking at a static image you could still determine "movement" (and therefore velocity) if a picture is blurred vs. absolutely sharp.

Additionally perspective, depth and shape of other objects might differ depending on velocity.

So it certainly could be possible to train a model based only on static images but it certainly should not outperform a model using both features of static images and differences between frames.

Edit:

I also looked at the repo and the presented method a bit. In the used method there could be a huge over-fitting problem if the video is only one single drive!

In this case the image detection would only identify "where" the picture was taken and since any point along the drive is associated with a single speed it could work from there i.e.: when the truck drove past a palm tree he was driving at 30 mph so identifying a palm in the picture should predict 30mph which works for a single drive data set but obviously is not able generalize.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.