1
$\begingroup$

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? which means it's connected to the neurons on the previous layer?

$\endgroup$
0

2 Answers 2

1
$\begingroup$

No, bias neurons are not connected to any previous neuron. This is visualized like this:

enter image description here

(source)

$\endgroup$
1
$\begingroup$

In a fully connected setting the bias shifts the weighted sum of the previous node output by a certain amount before applying the activation function.

In practice, it's a column vector b (bias [initialized as a constant vector]) added to the vector Wx (the product of weight matrix (W) and input vector (x)) as:

$$\mathrm{Layer2output} = W.\mathrm{Layer1output} +b$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.