Let's consider one has built a fully-supervised neural network for some task, e.g. localizing an object in various scenes. As you can imagine, it is quite time-consuming to label data: one has to manually localize the object in an image and draw a bounding-box around it - one at a time.
Say we have a normal convolutional neural network (CNN) for the fully-supervised localization, thus something like this:
2D input image | convolutional-layer 1 | ... | convolutional-layer N | ---flattened output--- | fully-connected classifier | output layer
How can we adapt this architecture to utilize weakly-labelled data, too?
No matter if we have a sliding-window approach, something like OverFeat or probability distributions for actually localizing objects: we always need fully-labelled training data. This is a problem, because labelling data fully-supervised is really time consuming. Hence, fully-labelled data sets are pretty rare.
In contrast to this, typically large amounts of weakly-labelled or unlabeled data exists. In my opinion there is a large potential in weakly-labelled / unlabelled data, due to its vast availability. The problem is using this potential without having to manually label every sample by hand.
That said, my question is: how can weakly-labelled data sets (i.e. "object" vs. "no object") or completely unlabelled data sets be utilized to improve the robustness of fully-supervised architectures, like the mentioned CNN?
Is it generally possible to mix a supervised approach with some unsupervised approach? Like enabling a fully-supervised architecture to somehow utilize weakly-labelled data for training?