I have been stuck in a Problem, for more than 2 weeks now.
I need to generate data set, that have actual Probability.
I am doing an experiment, my ML framework gives me probability intervals for the class labels (each sample). I want to check how many times the "actual Probability" is included in the Intervall.
I have tried Uniform distribution, and I have generated in the range [0,1) 4-dimensional data. I have summed it together and divided it by the number of the features. That's my actual probability. The actual probability is given to the Bernoulli distribution to get a class labels. But The actual probabilities are also uniform distribution, and it is supposed to. But Framework does not learn, almost all prediction values are 1. Example: I have created dataset using uniform distribution. 4 features + target
0.5507979, 0.93330316, 0.9868236, 0.533589, 1
0.2909047, 0.63284733, 0.6656879, 0.174750, 0
actual probability: (0.5507979+0.93330316+0.9868236+0.533589)/4 = 0.7511284
actual probability (0.7511284) given to Bernoulli distribution, gives me class label 1.
I have an idea that if I generate first the True Probability and according to that create feature values but I do not know how to do it. Example: let's say True probability (p) = 0.80, and most probably Bernoulli gives me a class label 1. Now how to generate some features so I can give them to ML Framework and compute the Intervall.
I am using Python.
your support will be highly appreciated.
Thank you very much in Advance.
