1
$\begingroup$

Suppose we have the following fully connected network made of perceptrons with a sign function as the activation unit, what issue arises, when trying to train this network with gradient descent? enter image description here enter image description here

$\endgroup$
2
  • $\begingroup$ What do you mean with "what issue arises"? Have you encountered issues when training such a network already? $\endgroup$
    – Oxbowerce
    Mar 13 at 21:22
  • $\begingroup$ @Oxbowerce This is an exercise from a past exam, which suggests that there is an issue. I have not used a network of this type for an actual tasks/ encountered an issue myself. $\endgroup$
    – WhatAMesh
    Mar 13 at 21:26
2
$\begingroup$

what issue arises, when trying to train this network with gradient descent?

The activation function is sign function or signum function (A little modified).
So, its Derivative will be 0 at all the points

Hence, the Gradient descent won’t be able to make progress in updating the weights and backpropagation will fail.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.