Consider a block of text with a variety of sentence types within it, of which there are 7. Within a text these sentences will be more or less likely to appear, dependent on where in the text they sit, and which sentence types have preceded them.
I'm looking into using Bayesian inference as an approach to working out the likelihoods of different sentence types appearing at certain points in the text, given where in the text they are and which sentence types have come before them.
The variable in question is the sentence type, and within Bayesian inference it is necessary to assume a prior distribution of the variable in order to precede, which is what I'm struggling with. Some sentence types will almost all be at the start or the end; others will be very unlikely to appear unless another type has come before it; and they have varying counts of how often they appear across a corpus of different texts, pictured below:
Any ideas on the problem above?
What are some more general thought processes to put to use in generating a sensible prior distribution?
If you would suggest a different approach to tackling this problem then please say.
This variable has some properties:
There are a few outcomes that are very likely, with the rest being a lot less likely but fairly uniformly distributed.
Some outcomes are a lot more likely depending on what has come before them, so the sentence structure matters
I'm guessing there are some probability distributions that would suit property (1) quite well, but property (2) I'm more confused by.