What thought processes do people use for generating priors on a variable's probability distribution?

Question

Example

Consider a block of text with a variety of sentence types within it, of which there are 7. Within a text these sentences will be more or less likely to appear, dependent on where in the text they sit, and which sentence types have preceded them.

I'm looking into using Bayesian inference as an approach to working out the likelihoods of different sentence types appearing at certain points in the text, given where in the text they are and which sentence types have come before them.

My problem

The variable in question is the sentence type, and within Bayesian inference it is necessary to assume a prior distribution of the variable in order to precede, which is what I'm struggling with. Some sentence types will almost all be at the start or the end; others will be very unlikely to appear unless another type has come before it; and they have varying counts of how often they appear across a corpus of different texts, pictured below:

Broader question

1. Any ideas on the problem above?

2. What are some more general thought processes to put to use in generating a sensible prior distribution?

If you would suggest a different approach to tackling this problem then please say.

Suggested solution

This variable has some properties:

1. There are a few outcomes that are very likely, with the rest being a lot less likely but fairly uniformly distributed.

2. Some outcomes are a lot more likely depending on what has come before them, so the sentence structure matters

I'm guessing there are some probability distributions that would suit property (1) quite well, but property (2) I'm more confused by.

Answer 1

In general the prior is simply calculated on the training data.

At first sight this looks like a simple sequence labeling problem. If so the model can be trained with HMM or CRF.