I would like to know if this idea has been tried before, and if so, where I can find more information about it.
This is an approach to generating artificial training data for segmentation tasks using an encoder-decoder network. However, instead of using a deep learning decoder, a non-deep learning generative algorithm (such as a physics or biology-based algorithm) would be used after the bottleneck. The generative algorithm would take as input only as many parameters as generated at the bottleneck. The loss function would be calculated as the difference between the input image and the generated image using an appropriate metric.
The advantage of this approach is that the classical generative algorithm would know the segmentation map of the output. The goal would be to overfit the encoder as much as possible to have a segmentation map training sample that is as close as possible to the input image.
I would appreciate any input or feedback on this approach.
