Labeling images for semantic segmentation can be expensive. Is it viable to train a model (such as Unet) to a good accuracy and then use this model to label more images to be used as further training data for the same model? Would this cause overfitting?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
I assume you're thinking of only using images where you are confident the model has segmented them correctly? I don't think this would cause overfitting - at least what we normally think of as overfitting. However, you could end up training the model to do even better on images where it already does well, at the expense of worse results where it is not doing so well (which I guess you could think of as a type of overfitting).
There is a technique called active learning that does something similar to this, though. Here you use the original model to identify images that would help improve the model the most, if they were labelled and added to the training set. These are then labelled by your domain experts and the model retrained. Obviously you can repeat this if need be until you stop seeing any improvement. See these blogs on active learning for more details: Active learning machine learning: What it is and how it works by DataRobot and Active Learning: Curious AI Algorithms on DataCamp
While I was writing this answer I found this article: Active Learning in Machine Learning Explained by Vatsal on Towards Data Science that suggests combining your approach and active learning.
$\begingroup$
$\endgroup$
It would not overfit, but you won't get a better classifier than the one you have used to label them.
Neural networks are Universal Approximators, thus the second classifier that you are going to train on that new labeled data, will approximate the function of the first NN (which itself was trying to approximate the "true" function)
If you train a model on the data that itself has labeled, it will obviously have 0 loss, thus 0 gradient, thus 0 changes in the NN