I'm planning to build a small car with autonomous driving (maybe modifying my current rc car or using a robot car kit, using arduino and raspberry).

I'll use a CNN, and I'm thinking how to collect data (I want to try a similar approach to the Udacity simulator). My doubt is if is better to aim at supervised learning or reinforcement learning.

I'm more inclined to the supervised learning, but I don't know the best way to record data. If I'll go with the robot car kit, the front wheel doesn't steer (it use different wheels speed to turn left or right). So I could create 3 function for each direction, like "soft left", "medium left" and "hard left" to control the car when I train it, and then predict from 7 possibility: - soft left, medium left, hard left, center, soft right, medium right, hard right So 7 possible output.

Otherwise if I use a car with the front wheel that are able to steer, I have 3 possible class, like -1 for left button, 0 for center (any button pressed) and 1 for right button. So 3 possible output, without any gradient for the steering angle.

With the reinforcement learning honestly I don't know where to start. How can I tell to the car when it does something wrong and give her a reward? With a simulator is simple, but with a real car how can I assing a reward score for each frame that I collect? If you have any paper about this I appreciate it.

What's the best approach in your opinion?

  • 1
    $\begingroup$ Nice try Uber, I'm not falling for that one again! Sincerely, Waymo. ;) $\endgroup$ – S van Balen Aug 9 '19 at 14:50
  • $\begingroup$ All jokes aside, I think this is a good place to start reinforcement learning: towardsdatascience.com/…, and this is a good resource to do self driving: github.com/SullyChen/Autopilot-TensorFlow $\endgroup$ – S van Balen Aug 9 '19 at 14:53
  • $\begingroup$ ahahah thank you @SvanBalen, I'll read the linked resource ;) $\endgroup$ – Nicola Aug 9 '19 at 14:56

~ If we consider the problem as an image classification task,

Basically, we can use a CNN trained on images belonging to 7 classes ( which represent the steering levels ). We give an image to the CNN and it outputs a steering level. This is a classification problem which will take each frame ( image ) of the video and output the steering level. So as the car steers the probability for the centre steering level will increase assuming that the road is straight after taking the turn.

See multiclass image classification.

~ else, if we train an agent for the problem ( reinforcement learning ),

Reinforcement learning requires an agent ( in our case, the car ) which produces some action ( steering levels ) in an environment ( street, road etc. ) and gets an reward accordingly. The rewards need to be maximized.

We will have a CNN which encodes an image to a vector representation.

So, consider two cases. We have an image which needs to the car to steer hard left. In the first case, the car turns soft left and loses its path. Here, the reward for this action becomes -100. In the other case, the car turns hard left, stays on the road and hence gets a reward of 100.

Basically, the agent gets a positive reward if it stays on the road doing some action ( steering level ) else it gets a negative reward showing that this action should not be made on such an image ( road ).

See here for learning RL.


You can try combining both the above methods. Generate class probabilities ( steering levels ) from CNN. If the car does not steer well, increase the loss by a certain factor.

Try thinking from a human perspective. Imagine driving a car and the cautions you take to avoid going out the road.

  • $\begingroup$ Thanks for the excellent answer! "Basically, the agent gets a positive reward if it stays on the road doing some action ( steering level ) else it gets a negative reward showing that this action should not be made on such an image ( road )." --> yeah, but how can I determine frame by frame if the car goes off the road and then assing her a lower reward? Or if the car hit an obstacle for example? Do I need sensors maybe? $\endgroup$ – Nicola Aug 9 '19 at 15:35
  • $\begingroup$ @Nicola maybe you can try out for sensors. I have lesser knowledge on this but I can surely say that the enviroment ( simulator ) will have some metric to understand that the car is off the road. $\endgroup$ – Shubham Panchal Aug 10 '19 at 0:15
  • $\begingroup$ Thank you for your advice! ;) $\endgroup$ – Nicola Aug 10 '19 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.