Prime model with mock data, good idea or not?

Question

I am currently working on text classifier with some pretty unique characteristics. The data is composed of about 2K categories but 98% of the data lives in just one of those 2K categories. However, our main problem is that we have a list of additional categories but have yet to see any transactions that fall into those categories (we know they will eventually).

Question:

1) Does it make sense to "prime" the model with these additional categories? For example, I could just add examples of these categories to my training set.

2) If priming is an acceptable are there any best practices or guides in doing so? I am wondering if I should just add the additional transactions to my training corpus, randomize them with other text, how many examples of the training set they should represent, etc..

Answer 1

This is not the answer you'll want to hear, but I would say, "no." Priming is not a good idea. A model is only as good as its inputs. If you're making up the inputs yourself, then your model isn't learning real patterns and you might as well just hard code a set of rules that represent how you would have created the primed data.

A fancier way might be to make a bayesian model where you create priors based on your assumptions. The bayesian model would then evolve as real data becomes available.