Skip to main content
added 517 characters in body
Source Link
Martin Thoma
  • 19.2k
  • 36
  • 95
  • 170

It's a binary semi-supervised classification problem. First, establish a base-line for the supervised case. Then try if the unlabeled data helps

Supervised

  1. From your labeled data: create a training, validation and test set. Don't touch the test set until the very end.
  2. It's a binary semi-supervised classification problem. FirstTry something simple, establish a base-line for the supervised casee. For example,g. a simple multilayer Perceptron (MLP) with 350 input nodes and 1 output node (giving the probability of "true").
  3. Try more stuff (e.g. https://github.com/MartinThoma/algorithms/blob/master/ML/mnist/many-classifiers/python.py#L92).
  4. Try to combine classifiers in ensembles (see examples).
  5. A (naive) bayes classifier might be worth being investigated.

Semi-supervised

  • You could train an auto-encoder with a bottleneck on the unlabeled data. Then remove everything after the bottleneck. Use this network as a preprocessing step. The idea is that this network finds a more meaningful abstraction of the relevant data. However, similar as PCA can have a very low projection error and still destroy the possibility to distinguish the classes.
  • SVMs can also be used in a semi-supervised setting.
  • You could try to find clusters in the unlabeled data + labeled data. Then you can get an a priori probability of "true" for each cluster (ignoring the unlabeled data). You could train very small models (overfitting!) on the data in the clusters.

Usually, you can interpret features. This might help to develop new features.

  1. From your labeled data: create a training, validation and test set. Don't touch the test set until the very end.
  2. It's a binary semi-supervised classification problem. First, establish a base-line for the supervised case. For example, a simple multilayer Perceptron (MLP) with 350 input nodes and 1 output node (giving the probability of "true").
  3. Try more stuff (e.g. https://github.com/MartinThoma/algorithms/blob/master/ML/mnist/many-classifiers/python.py#L92).
  4. Try to combine classifiers in ensembles (see examples).
  5. A (naive) bayes classifier might be worth being investigated.

Usually, you can interpret features. This might help to develop new features.

It's a binary semi-supervised classification problem. First, establish a base-line for the supervised case. Then try if the unlabeled data helps

Supervised

  1. From your labeled data: create a training, validation and test set. Don't touch the test set until the very end.
  2. Try something simple, e.g. a multilayer Perceptron (MLP) with 350 input nodes and 1 output node (giving the probability of "true").
  3. Try more stuff (e.g. https://github.com/MartinThoma/algorithms/blob/master/ML/mnist/many-classifiers/python.py#L92).
  4. Try to combine classifiers in ensembles (see examples).
  5. A (naive) bayes classifier might be worth being investigated.

Semi-supervised

  • You could train an auto-encoder with a bottleneck on the unlabeled data. Then remove everything after the bottleneck. Use this network as a preprocessing step. The idea is that this network finds a more meaningful abstraction of the relevant data. However, similar as PCA can have a very low projection error and still destroy the possibility to distinguish the classes.
  • SVMs can also be used in a semi-supervised setting.
  • You could try to find clusters in the unlabeled data + labeled data. Then you can get an a priori probability of "true" for each cluster (ignoring the unlabeled data). You could train very small models (overfitting!) on the data in the clusters.

Usually, you can interpret features. This might help to develop new features.

Source Link
Martin Thoma
  • 19.2k
  • 36
  • 95
  • 170

  1. From your labeled data: create a training, validation and test set. Don't touch the test set until the very end.
  2. It's a binary semi-supervised classification problem. First, establish a base-line for the supervised case. For example, a simple multilayer Perceptron (MLP) with 350 input nodes and 1 output node (giving the probability of "true").
  3. Try more stuff (e.g. https://github.com/MartinThoma/algorithms/blob/master/ML/mnist/many-classifiers/python.py#L92).
  4. Try to combine classifiers in ensembles (see examples).
  5. A (naive) bayes classifier might be worth being investigated.

Usually, you can interpret features. This might help to develop new features.