I am aware of the existence of semi-supervised learning approaches, such as the Ladder Network, where only a subset of the data is labeled. Are there any methods or papers which consider correctness probabilities for the labels of that training data subset? That is, some labels may be correct with 100% probability, while others may have only 70% or 45% probability of being correct. Any links to papers or work in this direction are highly appreciated.

  • 2
    $\begingroup$ Just adjust the loss function? For example, with a binary classifier a probability of $p$ in one class induces a probability of $1-p$ on the other. In a multiclass setting you have to decide how to apportion the complement (equally?). With regression losses it's just a multiplicative coefficient. $\endgroup$ – Emre Jun 9 '17 at 16:24
  • $\begingroup$ Would simply adjusting the target work? If the probability of correctness is 40%, you could assume that the other classes have in sum a probability of 60%. Hence your target would not be 1,0,0,0 but 0.4,0.2,0.2,0.2, for example. $\endgroup$ – Martin Thoma Jan 1 '19 at 21:59

I do not know any papers, I would be greatly appreciated if someone would link some.

In my case, I always test by un-labelling my already known labelled Data by using a "traditional" 67% - 33% (train - test) split and checking how the labelling performs in various metrics (Accuracy, Logloss, etc.).

Moreover, there are various categories of Semi-Supervised Learning. For instance, if you would use Active Learning (pool based approach, includes incrementally sample/s each time), you could view how the performance ranges.

Finally, you may use as well Cross-Validation in the same sense for hyper parameter tuning for your Semi-Supervised Learning algorithm. All in all, in my perspective Semi-Supervised Learning, has a lot yet to offer and your question may be rather more domain, data specific and visualsing the Data and Clustering may give you new insights.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.