A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

# Questions tagged [backpropagation]

Use for questions about Backpropagation, which is commonly used in training Neural Networks in conjunction with an optimization method such as gradient descent.

198 questions
Filter by
Sorted by
Tagged with
23 views

### Interpreting Gradients and Partial Derivatives when training Neural Networks

I am trying to understand of purpose of partial differentiation in NN training by knowing how to interpret gradients and their partial derivatives. Below is my way of interpreting them so I would like ...
18 views

### Justification for values used in backpropagation

I'm learning the method for backpropagation in adjusting weights. A generalization of a formula used to determine the change made to a respective weight is where is the rate the total error changes ...
28 views

### Implementing “full convolution” to find gradient w.r.t the convolution layer inputs

I've been trying to implement "full convolution" w.r.t to convolution layer inputs. According to this article, it looks like this: So, I wrote this function: ...
36 views

### Forward pass vs backward pass vs backpropagation

As mentioned in the question, i have some issues understanding what are the differences between those terms. From what i have understood: 1) Forward pass: compute the output of the network given the ...
24 views

### backpropagation between fully connected layer and convolution layer?

This is a simple example of a network consisting of two convolutional layers and one fully connected layer. ...
13 views

### Backpropagation through the inputs of convolutional layer in LeNet

I am trying to understand backprop for LeNet according to the original article http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf I think I have successfully done backdrop till the C3 ...
29 views

### Why multiply by 2 when calculating partial derivatives during backpropagation?

I'm wondering why we multiple by 2 when calculating partial derivatives. I'm referencing the 2's that I've circled below, from here. We also see this in the python implementation, ...
52 views

### Hello, anyone able to direct me to a “cheat sheet” of Neural Network equations with legends?

Can anyone here can direct me to a site that provides a cheat sheet of equations for Neural Networks with a legend for the notation? It can be on any and all aspects of NN, be it forward or back ...
10 views

### Question regarding: Vectorization Math of Backpropagation in a Neural Network

Formula: These are the formula I use for backpropagation from Brilliant: Question: If we consider a Neural Network with the structure (3,2): And we would start calculating the derivative (for 1 ...
34 views

### Problem with chain rule in softmax layer when differentiated separately

I have some problems with backpropagation in softmax output layer. I know how it should work but if I try to apply the chain rule in the classical way, I get different results compared to when Softmax ...
22 views

### How to do Back Propagation Updation for below code?

Below is the code by Siraj Raval for implementing Neural Networks from Scratch. I have some doubts regarding the Code: Why during updation he did W2 = W2 + L1.T.dot(L2_Delta). I Mean shouldn't it be ...
33 views

106 views

### Neural Network from scratch: cost increasing over epochs

I'm trying to design a neural network from scratch. After training my neural network, I make a plot of the cost vs epochs, which I would expect to decrease throughout the runtime of the NN training, ...
15 views

### Backprop: backward pass way faster than forward pass

I started to work with my own implementation of backpropagation algorithm, that I made five years ago. For each training sample (input-output pair), I make a forward pass (to compute outputs of each ...
34 views

### Doubt in Derivation of Backpropagation

I was going through the derivation of backpropagation algorithm provided in this document (adding just for reference). I have doubt at one specific point in this derivation. The derivation goes as ...
15 views

### What's the input for the cost function?

I'm trying to implement deep Q-learning, but I do not know what to put into the cost function. My net has 8 scalar inputs, 4 scalar outputs (from 0-1) and no hidden layers. To calculate the cost I ...
73 views

### Back-propagation and stochastic gradient descent

Is backpropagation a learning method or an optimisation method? How are backpropagation and stochastic gradient descent related to each other?
26 views

### Backpropagation chain rule example

My question is in regards to an MIT course example. The instructor delves into the backpropagation of this simple NN. I have two questions. Why do we seem to disregard the weights of the second ...
10 views

### MADALINE learning vs modern backpropagation

I am just a tad bit confused from my reading. If we have a multiple layer ADALINE NN (MADALINE) (or perceptron even); how would this have been trained prior to backpropagation? If I am correct, with ...
28 views

21 views

### What does this expression from gradient descent mean?

I am looking over some neural network theory and came across this equation, coupled with this description (gradient descent ball-valley analogy): ''let's think about what happens when we move the ...
62 views

### Having problem in back propagation part for dimension

I was trying to build a neural network with single hidden layer from scratch. In back propagation part some problems have raised. For calculating gradient of loss function with respect to weight in ...