I'm building a convoluted neural network to teach a toy car, powered by a Raspberry Pi, how to drive based on incoming streams of frames from a webcam mounted on top of the car. The top half of each image is irrelevant. What matters is the curvature of the road, and this is in the bottom half.
I've generated a substantial amount of data (about 40k records) by driving the car around myself and recording what I do (commands are left, right, and straight) and what the frames are. However my trained ConvNets aren't giving me the performance I'd hoped for. My experimentation with one of the deployed models confirms that it is indeed tricked by changes in the top half of the streaming images.
A simple solution is to programmatically cut the frames in half so that the neural net only receives relevant portions. However, deep neural nets are praised for their ability to learn features with (ideally) zero human future transformations, so I want to avoid this approach. This project is meant to be a learning experience for me so I want learn how to architect the ConvNet more effectively. Each training session takes several hours to run, so rather than try everything under the sun, I'd figure I'd reach out to the community here to narrow my focus of exploration.
One thought I have is to put a fully connected (FC) layer in front so that subsequent layers effectively only convolve over the relevant portion of the image. I think this could potentially work if this up front FC layer learned to assign very small weights to pixels in the top half of the image. Could this work? Are there better architectures?