In course cs231n, I need to implement backward pass computation for an affine (linear) layer:
def affine_backward(dout, cache): """ Computes the backward pass for an affine layer. Inputs: - dout: Upstream derivative, of shape (N, M) - cache: Tuple of: - x: Input data, of shape (N, d_1, ... d_k) - w: Weights, of shape (D, M) Returns a tuple of: - dx: Gradient with respect to x, of shape (N, d1, ..., d_k) - dw: Gradient with respect to w, of shape (D, M) - db: Gradient with respect to b, of shape (M,) """ x, w, b = cache dx, dw, db = None, None, None
I do not understand why the shape of
(D, M), as the output of the layer is a matrix
(N, M) - N being batch size.
This would only make sense if the output is a scalar.
What am I missing?
Thanks in advance.