4
$\begingroup$

Bishop's Perceptron loss

On one hand, it is stated in equation 4.54 of Chris Bishop's book (pattern recognition and machine learning) that the loss function of perceptron algorithm is given by:

$${E_p}(\mathbf{w}) = - \sum\limits_{n \in M} {{\mathbf{w}^T}{\phi _n}} {t_n}$$

where $M$ denotes the set of all misclassified data points.

Original Perceptron loss

On the other hand, The loss function used in the original perceptron paper written by Frank Rosenblatt is given by (wikipedia):

$${1 \over s}\sum\limits_{j = 1}^s {\left| {{d_j} - {y_j}\left( t \right)} \right|} $$ which when translated to the notation of Bishop's book is given by:

$${1 \over N}\sum\limits_{n = 1}^N {\left| {{t_n} - {\mathbf{w}^T}{\phi _n}} \right|} $$ where $N$ denotes the set of all data points.

My question

My question is that why Bishop's version of Perceptron loss is different from the original paper? Considering Bishop's book as a highly recognized book in machine learning field, can we call it Bishop's Perceptron?!

Scikit-learn's implementation

By the way, it seems that Scikit-learn uses Bishop's version of Perceptron loss (Scikit-learn documentation). It is apparent from the following formula and figure:

-np.minimum(xx, 0)

which for one sample reduces to: $$ - \min \left( {0,{\mathbf{w}^T}{\phi _n}{t_n}} \right)$$

enter image description here

$\endgroup$
4
$\begingroup$

I managed to find the Bishop's version by unearthing! the Rosenblatt's 1962 Principles of neurodynamics, Page 110 book, so the Wikipedia's version must be the alternative one.

It is worth noting that the book also has a chapter on error back-propagation (page 292), which resembles the Wikipedia version, but I think it is not exactly the same.

| improve this answer | |
$\endgroup$
  • 3
    $\begingroup$ Thanks a bunch for your detailed and demanding response. I’m really surprised by the amount of search you did in order to fix the issue. $\endgroup$ – pythinker Apr 12 '19 at 20:45
  • $\begingroup$ @pythinker My pleasure! I'm glad I could help. This detective stuff is fun ;) $\endgroup$ – Esmailian Apr 12 '19 at 20:49
  • 1
    $\begingroup$ I’m pleased to hear that you are interested in this type of questions. So I will try to pose more questions like this :) $\endgroup$ – pythinker Apr 12 '19 at 20:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.