How to determine the correct target for classification probability when the observed samples are probabilities of each class?

Question

I have data in which each event's outcome can be described by a probability of a categorical occurrence. For example, if all of the possible class outcomes are A, B, C, or D suppose in one event 7/10 people selected category A, 2/10 selected category B, and 1/10 selected C.

The goal of the model is to predict the probability of each class as close to the observed truth as possible. However, it is unclear how to determine the proper target. Here are the options I've thought about doing:

1. The target becomes the predominant class selected. That is, in the event described above A=1, B=0, C=0, D=0.

2. Additional classes/categories are created and then become bins of all possible observed probabilities. For example, the event above would become A=1 when Pr(A) >= 0.7.

I THINK option 1 is the the best as option 2 would result in a lot of categories and spread. However, when considering verification of the model output, i.e. Y = {A=0.9, B=0.1, C=0, D=0} for each sample would it not be advantageous to use the actual observed probabilities? Are there other options for the target?

Answer 1

Imho the "cleanest" option would be to train a probabilistic model on the original categorical target, then obtain the predicted probabilities for every category as the final "predictions". By "training on the original target" I mean designing each instance as an event, e.g. in order to represent that 7/10 people select category A there would be 7 instances where the target is category A out of 10 instances in total.

• The most simple option is Naive Bayes, but depending on the data it tends to always predict extreme probabilities, which would defeat the purpose.
• An ad hoc Bayesian model could give very good results but it's probably more work to design it, depending on the features.