Strategies for handling unlabeled data which is slightly different from the labeled data

Question

Suppose you have a dataset with the following properties:

1. The number of samples is fairly large (~100K samples)
2. There are ~150 contextual features and 1 feature consisting of a text-string (which can, of course, be split into any number of features depending on the pre-processing of the text). It is expected that the text-string will have really great predictive power
3. Samples are divided into 3 categories (prior to you receiving the data) based on a few of the contextual features with category A containing ~5% of the samples, category B containing ~20%, and category C containing the remaining 75%
4. Category A is entirely labeled, category B is partly labeled (with only a small proportion being unlabeled), and category C is entirely unlabeled
5. The features used to categorize the samples are likely to influence the probability of a sample belonging to class 0 or class 1.
6. The samples are not completely different between categories (that is to say, we're not talking cats versus dogs). E.g.: Two very similar samples might end up in different categories based on very small differences on a numerical feature with a large range

The purpose is to build a classifier which will correctly classify the samples. That might look like a semi-supervised learning-problem, but I am worried about the structural differences between the categories. Hence my question: Which strategies could be employed to build a classifier performing well on all of the samples?

Of course I could just be conservative and only deal with the labeled data, but there is great value in also being able to predict the unlabeled data (e.g. the 75% of the data in category C). That's why I'll try to pick your brains for creative solutions!

Answer 1

There is no straightforward way to deal with this scenario, but here are some ideas:

• Augment the A/B/C label set with an extra label "U" for "unknown". Now all samples are labeled.

• Predict A/B/C labels from the unlabeled samples from the other pieces of data in the sample. Now all samples are labeled.

• Train a system for data with labels and another system for data without them. Use them according to the availability of the label. It would be great if your ML approach allows for sharing parts of the model for labeled data and the one for unlabeled data (e.g. neural networks).