1
$\begingroup$

I'm trying to add a bias neuron to my neural network that uses the backpropagation algorithm. I'm trying to figure out how I should go about this, should I treat the bias neuron as a regular neuron? which means it's connected to the neurons on the previous layer?

Improve this question
$\endgroup$
1
  • $\begingroup$ Please, consider upvoting the answers if you found them useful, and marking one as correct if deemed so. Alternatively, please considering describing what the answer is lacking or why you think it is not correct, so that it can be improved. $\endgroup$
    – noe
    Dec 24, 2020 at 13:47

2 Answers 2

Reset to default
1
$\begingroup$

No, bias neurons are not connected to any previous neuron. This is visualized like this:

enter image description here

(source)

Improve this answer
$\endgroup$
1
$\begingroup$

In a fully connected setting the bias shifts the weighted sum of the previous node output by a certain amount before applying the activation function.

In practice, it's a column vector b (bias [initialized as a constant vector]) added to the vector Wx (the product of weight matrix (W) and input vector (x)) as:

$$\mathrm{Layer2output} = W.\mathrm{Layer1output} +b$$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.