To be clear, do you think the data could be distributed according to a particular probability distribution? If so you should model it directly without a neural network. Your model parameters are simply the parameters of that probability distribution ($\mu$ and $\sigma^2$ for a gaussian, $\lambda$ for a poisson etc.,).
If you think your data is distributed according to a distribution with parameters conditioned on some input features you should have a model (for example a linear model, neural network etc.,) output the parameters of that probability distribution.
In either of these cases, you could then take a fully Bayesian approach (See, for example, chapter 5.3 of Machine Learning A probabilistic Perspective K. Murphy), but a simpler approach would be to perform Maximum Likelihood Estimation. The calculations for Maximum Likelihood Estimation are straightforward and readily available for common distributions. For a Gaussian you simply calculate the sample mean and variance and those are your model parameters. If you are outputting the distribution parameters from a neural network you will need to set your loss as the negative log likelihood associated with that particular distribution. Perform Maximum Likelihood Estimation for all models you are considering, and then compare each of the model's likelihood under a held out validation set. The likelihood is calculated as the product of probabilities of each data sample under that model. The distribution that fits the data the best is the one which the has highest likelihood on the held-out validation set.