0
$\begingroup$

In a typical supervised learning setting with a few positive and a few negative examples, it is clear that unlabeled data carries some information that can benefit learning and that is not captured in the labeled data. For example one can estimate mean values, bounds and some other geometrical characteristics of the data-set with much higher precision if you do not discard the (massive) unlabeled data.

On the other hand, the most common ML algorithms from Neural Networks to SVM do not take advantage of this information (at least in their standard, most common form). My question:

  • Is there any theoretical framework where unlabeled data is treated in the supervised setting?

I can think of semi-supervised ways to approach this (first cluster and then label the clusters). Are there any other?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
0
$\begingroup$

In a neural network model, you can use autoencoders.

The basic idea of an autoencoder is to learn a hidden layer of features by creating a network that simply copies the input vector at the output. So the training features and training "labels" are initially identical, no supervised labels are required. This can work using a classic triangular network architecture with progressively smaller layers that capture a compressed and hopefully useful set of derived features. The network's hidden layers learn representations based on the larger unsupervised data set. These layers can then be used to initialise a regular supervised learning network to be trained using the actual labels.

A similar idea is pre-training layers using a Restricted Boltzmann Machine, which can be used in a very similar way, although based on different principles.

Improve this answer
$\endgroup$
1
  • $\begingroup$ @{neil-slater} thanks for the input! What doesn't feel quite right (to me) in these Deep Learning Examples, is that they are more of a recipe: yes, you train an autoencoder to simplify the data and you also use labeled data to train the DLN. It's true that autoencoder trick "helps", but it doesn't appear to be theoretically necessary: it's just harder to train without. $\endgroup$
    – Tnatsissa H Craeser
    Commented Dec 3, 2015 at 1:17
0
$\begingroup$

You already dropped the key word "semi-supervised" in your question. Indeed, semi-supervised learning is the answer to your question. Search for this term with your favourite search engine or library catalog to find out about algorithms for semi-supervised learning.

Improve this answer
$\endgroup$
1
  • $\begingroup$ @knappen Yes, I do believe that this is viable approach. However I'm guessing that for example in PAC (Probably Approximately Correct) context when one learns "something", then knowledge of the underlying distribution of this "something" can be very useful and this distribution can be derived from unlabeled data. So, I'm guessing there must be a different and perhaps more precise way of using unlabeled data than clustering. For example, clustering by itself is oftentimes problematic. $\endgroup$
    – Tnatsissa H Craeser
    Commented Nov 25, 2015 at 15:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.