# Applying bayesian methods to a simple neural network

This is a really simple neural network with backprop. If one had to apply bayesian "inferences" to update the weights and biases, what would change in the code.

#Forward Propogation
hidden_layer_input1=np.dot(X,wh)
hidden_layer_input=hidden_layer_input1 + bh               # linear transformation
hiddenlayer_activations = sigmoid(hidden_layer_input)     # non-linear transformation
output_layer_input1=np.dot(hiddenlayer_activations,wout)
output_layer_input= output_layer_input1+ bout             # linear transformation
output = sigmoid(output_layer_input)

#Backpropagation
E = y-output
slope_output_layer = derivatives_sigmoid(output)
slope_hidden_layer = derivatives_sigmoid(hiddenlayer_activations)
derivative_output = E * slope_output_layer
Error_at_hidden_layer = derivative_output.dot(wout.T)
derivative_hiddenlayer = Error_at_hidden_layer * slope_hidden_layer
wout += hiddenlayer_activations.T.dot(d_output) *lr
bout += np.sum(derivative_output, axis=0,keepdims=True) *lr
wh += X.T.dot(d_hiddenlayer) *lr                          # update weight
bh += np.sum(d_hiddenlayer, axis=0,keepdims=True) *lr     # update bias


If my understanding is correct of using bayes method to derive the weights and biases, it would help in getting to those values faster, leaving behind unnecessary loops for weights which would not converge.

Apart from that, coding and observing it is the best bet to get a real perception.

Would like an understanding.

• It's not that simple, and I don't know what you mean by "coding and observing it is the best bet to get a real perception". If you want to go Bayesian you have two options: variational- and Monte Carlo methods. Are you familiar with them? I suggest implementing Bayesian linear regression before trying a neural network.
– Emre
Jul 13, 2017 at 19:55
• I think this question is incomprehensible. For starters, there are no loops in the code, so I don't understand why the user asks for "leaving behind unnecessary loops".
– knb
Aug 13, 2017 at 16:45