I would like to know when to stop doing semi supervision? For example, if I learn a classifier from a small dataset and then use it to label a pool of unlabelled dataset. I then use the newly labelled data to train my classifier again. How long should this process be continued?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
4
Most of the semi-supervised methods are heuristics and more or less are modifications of the standard supervised learning algorithms, where you are trying to take into account unlabeled data considering a small dataset of labeled data. If the data from these two datasets do not follow the same distribution, then you have to look for transfer learning methods.
I guess that you are asking for a stopping criterion for iterative methods/approaches of semi-supervised learning, otherwise you do not need a stopping criterion. It depends on the approach. You could define a threshold that reflects the confidence that the unlabeled data could be labeled correctly and use them as pseudo-labeled instances with the ones that are already labeled for training an accurate classifier. Another option could be to define a maximum number of iterations.
answered May 2, 2019 at 8:51
-
$\begingroup$ Thank you. You correctly get what I wanted to know. I have also seen in the literature about the solutions that you mention here, but I am not sure what does this confidence score means in terms of a classifier. Is there a family of classifier can provide this score, or I can get such a score from any classifier. Any example of a sample classifier confidence score and its use in semi-supervision is highly appreciated. $\endgroup$– N. F.Commented May 2, 2019 at 17:39
-
$\begingroup$ It depends on the algorithm that you are going to apply and particularly the way that you are going to optimize the objective function that reflects the similarity among the instances of the two datasets. There are ways to select instances or weight them considering the similarity. One criterion could be if you select the same instances in two sequential iterations or if the weights stay unchanged. Also, you can check the E-M algorithm for semi-supervised learning. $\endgroup$ Commented May 2, 2019 at 21:43
-
$\begingroup$ Using sklearn, how do you get the weights ? I have the same problem doing a semi supervised learning based on an iterative random forest model. E-M algortihm could be a solution but it's not implemented in sklearn , only as a part of GMM. $\endgroup$ Commented Oct 26, 2019 at 10:27
-
$\begingroup$ You are able to compute the weightes externally and pass them to sklearn functions that support sample weights. $\endgroup$ Commented Nov 3, 2019 at 18:25