I am trying to reason through how this works from the beginning of the neural network model to the end from a high level and I want to make sure my understanding is correct, I have probably misunderstood something somewhere....

  1. Start with random weight initialization in the network, at this point, bayes theorem looks something like this, since we have to start somewhere, and the random weights should have a mean of zero...

$$ p(\theta | X) \propto p(X|\theta)p(\theta) \text{ where } p(\theta) \sim \mathcal{N}(0, 1) $$

  1. Minimize the error for a given number of iterations, add some gaussian noise to all the weights and then minimize them again.

  2. Do step 2 repeatedly for a fixed number of iterations.

  3. End up with a discrete set of model weights that approximate the distribution of possible weights

  4. Make predictions with all of the weights, which are weighted according to their distance from the mean

  5. Take the mean and variance of the predictions as the final prediction which has the mean as the prediction and the variance as the uncertainty.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.