# Find partial derivative of softmax w.r.t logits in python

I have trouble implementing back propogation for multi class classification

My neural network has 2 layers

# Forward propagation

X -> L1 -> L2

weights W are initialized as random

np.random.randn(this_layer_units, previous_layer_units) * 0.01


X is input of size (no_features * number of examples)

Z1 = (w1 * x) + b1

A1 = relu(Z1)


L1 has ReLu activation

Z2 = (w2 * A1) + b2

A2 = softmax(Z1)


L2 has softmax activation

cost is caluclated using cross entropy

cost = -(1/m)*np.sum((Y * np.log(A2) ) + ((1 - Y)*np.log(1-A2)))


# Back propagation

derivative of cost is calculated

dA2 = -(1/m)*(np.divide(Y, A2) - np.divide(1 - Y, 1 - A2))


dA2 = derivative of cost w.r.t softmax

Y = one hot encoded True values

softmax is

np.exp(z)/ np.sum(np.exp(z))


now, how do i find dZ2 (derivative of Z2) w.r.t dA2 (softmax activation)

Link to entire jupyter notebook code

• Backprop is never Specific to a dataset.. Here's mine on MNIST github.com/AdityaSoni19031997/Machine-Learning/blob/master/… – Aditya Sep 23 '18 at 17:44
• It won't be softmax of Z1 though.. Also your derivative calculation isn't correct imho.. Here we ain't doing normal Scaler derivatives... Everything is Vector, so things change a bit... Try reading blogs to quench your Knowledge base... – Aditya Sep 23 '18 at 17:47