I would like to know when to stop doing semi supervision? For example, if I learn a classifier from a small dataset and then use it to label a pool of unlabelled dataset. I then use the newly labelled data to train my classifier again. How long should this process be continued?
Most of the semi-supervised methods are heuristics and more or less are modifications of the standard supervised learning algorithms, where you are trying to take into account unlabeled data considering a small dataset of labeled data. If the data from these two datasets do not follow the same distribution, then you have to look for transfer learning methods.
I guess that you are asking for a stopping criterion for iterative methods/approaches of semi-supervised learning, otherwise you do not need a stopping criterion. It depends on the approach. You could define a threshold that reflects the confidence that the unlabeled data could be labeled correctly and use them as pseudo-labeled instances with the ones that are already labeled for training an accurate classifier. Another option could be to define a maximum number of iterations.