I'm working on a convolutional neural network that should predict up to 3 (x,y) coordinate pairs representing the waypoints of a concrete path, given an input image. This network will be used to help guide a robot along a pathway. The issue is that my network's "accuracy" is peaking at around 90%, where I am using L1 loss to calculate the loss between the predicted (x,y) coordinates and the actual (x,y) waypoints.
As an aside, I only have about 4,500 input images to train with, as I have to manually record video, grab image frames from the video, and hand label them by labeling what I deem the waypoints are for the images. I know the dataset is not very large, and realize that is a big factor, but I use augmentation to help with that a bit.
That aside, at first I thought it might be my model (which it could still be), so I tried to use a pre-trained AlexNet with a few dense layers at the end to see if the accuracy would be vastly different. As I suspected, the loss was about the same (a little worse, but to be expected when using an older model). My next thought was that my labels for my images may be inconsistent. I was hoping that some more experienced people might be able to lend some guidance for more consistent rules to label my data.
Some sample images from my dataset are the following (not able to embed, sorry):
- Image with 2 waypoints
- Image with 3 waypoints
- Image with 2 waypoints and bounding lines
- Image with 2 waypoints and bounding lines
The first two images just show the waypoints on the images, while the second two show the waypoints along with some bounding lines I draw on when labeling (not passed as input to the model) to help with some of my rules. When labeling images, the top 25% of the image is above the horizontal blue line, while the bottom 25% of the image is below the horizontal red line. The middle 50% of the image is the space between the two lines
Currently, my rules are the following:
- Waypoint must be within top 75% of the image (stated in another way, above the bottom 25% of the image)
- If there is a fork in the road within the top 25% of the image, place a single waypoint in the center of them (when we get closer to the fork, then mark them as individual waypoints)
- If there are more than 3 waypoints in the image, place a waypoint between several that are farther away
- When there is a fork in the middle 50% of the image, mark all individual waypoints
I mostly adhere to these rules, but there are some exceptions as not all images fit into this tight box.
TL;DR: Based on the images above, I would greatly appreciate help changing or coming up with more consistent rules to help label my data for waypoint regression, so that I am able to increase model accuracy/decrease loss.
