1
$\begingroup$

The criterion for firing a perceptron is as follows

Firing Criterion

Why is it that when the function $w \cdot x + b = 0$ the output is zero as well. Why couldn't it have been set to 1?

If one were to simulate the behavior of the perceptron with a sigmoid function, $w \cdot x + b$ can be multiplied by any arbitrary constant $C>0$ (as $C$ tends to infinity) as long as $w \cdot x+b$ is not equal to 0. This is because at 0 condition, the firing always gives output as 1 instead of zero it should have giving. However, changing the firing rule overcomes this problem. Then, is there any reason why 0 should be the output when $w \cdot x+b= 0$?

$\endgroup$
1
  • $\begingroup$ I think it makes no difference because $wx+b$ will almost never be exactly zero. Remember that all these variables are real numbers. Welcome to the site! $\endgroup$
    – Emre
    Aug 28 '17 at 20:15
1
$\begingroup$

The perceptron algorithm was invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt (Wikipedia EN)

It was the first neural network like architecture. It was thought that (a combination of) perceptrons could learn anything. There was a lot of controversy about this and after a while Marvin Minsky showed that a single perceptron could not solve the XOR problem. This was the end of the first neural network hype.

After a couple of decades the multilayer perceptron was invented, and with it the backpropagation algorithm. The backpropagation algorithm can learn weights that are not directly connected to the output unit. However, it needs a differentiable activation function. Hence, the ramp activation function was not sufficient anymore and others like the sigmoid and tanh were used for this.

Besides, the perceptron was first implemented using electronic boards and not with the software available to us today. Binary thresholds made this complicated process a lot easier.

So, in short Rosenblatt could have chosen for any version of the heaviside step function but he chose for the one where $w \cdot x + b = 0$ leads to a zero output. If you think binary values this makes sense, a zero input (i.e. False) leads to a zero output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.