All Questions
4 questions
0
votes
0
answers
149
views
calculating derivative of bias in backpropagation
Looking at the algorithm in wikipedia, we can implement backpropagation by calculating:
$$\delta^{L}=\left(f^{L}\right)'\cdot\nabla_{a^{L}}C$$
(where I treat $\left(f^{L}\right)'$ as an $n\times n$ ...
- 101
0
votes
1
answer
67
views
Vanishing gradient problem
In a neural network, does gradient vanish during a great number epochs as well, rather that only vanishing through different layers?
2
votes
1
answer
531
views
Gradient passthough in PyTorch
I need to quantize the inputs, but the method (bucketize) I need to do so is indifferentiable. I can of course detach the tensor, but then I lose the flow of gradients to earlier weights. I guess ...
- 203
4
votes
3
answers
12k
views
Forward pass vs backward pass vs backpropagation
As mentioned in the question, I have some issues understanding what are the differences between those terms.
From what I have understood:
Forward pass: compute the output of the network given the ...