I'm new to CNN and trying to study some MATLAB sample codes (cause I need to know the internal calculation). I recently realized that the sample code I'm using doesn't multiply error by sigmoid's derivative in back propagation. The feed forward process has sigmoid as last layer's activation function so from my understanding, back propagation error = (outputs - target) * sigmoid's derivative(outputs). However, the author intentionally disabled this multiplication with the following code:
if cnn.loss_func == 'cros'
if cnn.layers{cnn.no_of_layers}.act_func == 'soft'
cnn.CalcLastLayerActDerivative = 0;
elseif cnn.layers{cnn.no_of_layers}.act_func == 'sigm'
cnn.CalcLastLayerActDerivative = 0;
end
end
When cnn.CalcLastLayerActDerivative = 0, error is defined just as (outputs - target). I tried to initialize cnn.CalcLastLayerActDerivative = 1 so that sigmoid's derivative is considered in back propagation but then I got worse error rate. I'm not sure whether it's just because sigmoid's derivative is in the range [0,0.25] or I'm not understanding back propagation correctly. Does anyone know why this is happening and whether I should add sigmoid's derivative in my calculation?
Thanks!