In this case I have a challenge of inspecting a recently mounted product, using computer vision, to detect the absence of any component that I have to check.
For this task, I tried the concept of detection of regions of interest, which in this case are the components I want to check. I moved to an approach using tensorflow's object detection API, which I would train a model to detect the components I wanted to check using annotated images. In this case, all images contained just the presence of the component.
The results were very good using validation images, but unfortunately produces a lot of false positives (cases that the component is not present but it still writes a box on its area), sometimes with 99%+ confidence (so discarding my chance to filter these cases) and, on other side, there's cases that it can't find at all a, for an human view, very clear component.
With these results, I tried to train the model with both positive and negative cases (presence/absence), hoping that it would learn the difference between them. Again, the false positives appeared, this time with a smaller frequency, but considering that my validation set is very small (about 10 images), in a global situation this error frequency can be very high, thus discarding the viability to use the model in production.
After several iterations of experiments, I'm starting to notice that the model is very unstable on its results to use in production. Is this approach the best way to solve this type of problem?
Unfortunately I don't have many examples to show the problem I'm facing at the moment, but just to illustrate my problem, here's two results of the approaches I listed:
I already tried using all models supported by the api, which includes several types of faster_rcnn and ssd. The one I stick with, with the best trade of training time and results, is the faster_rcnn_resnet101 with COCO training as the pre-training.