The author explains that for for a single neuron in neural net, by choosing cost function as cross entropy we can eliminate the derivative of activation function in the gradient term, if the activation function is choose to be sigmoid function.
In the problem, he asks why we cannot eliminate the input to neuron term x in gradient of cost function with respect to weights.
I had the following reasoning, in order to compute gradient of Cost, we use chain rule, and derive cost with respect to activation, and activation with respect to (w * x + b), and the sum with weight.
For a weight Wi, the derivative of (Summation w * x + b) w.r.to Wi is always xi, and the derivative of activation function cannot know that, so it can never eliminate Xi unless its zero.
Or is there any other subtle reasoning?