1
$\begingroup$

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? which means it's connected to the neurons on the previous layer?

$\endgroup$
1
  • $\begingroup$ Please, consider upvoting the answers if you found them useful, and marking one as correct if deemed so. Alternatively, please considering describing what the answer is lacking or why you think it is not correct, so that it can be improved. $\endgroup$
    – noe
    Dec 24 '20 at 13:47
1
$\begingroup$

No, bias neurons are not connected to any previous neuron. This is visualized like this:

enter image description here

(source)

$\endgroup$
1
$\begingroup$

In a fully connected setting the bias shifts the weighted sum of the previous node output by a certain amount before applying the activation function.

In practice, it's a column vector b (bias [initialized as a constant vector]) added to the vector Wx (the product of weight matrix (W) and input vector (x)) as:

$$\mathrm{Layer2output} = W.\mathrm{Layer1output} +b$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.