I was revisiting perceptron learning algorithm. The wikipedia page gives the algorithm as follows:

  1. Initialize the weights to 0 or a small random value.
  2. For each example $j$ in our training set $D$, perform the following steps over the input $x_j$ and desired output $d_j$:
    a. Calculate the actual output: $$ \begin{align} y_j(r) &= f(w(t).x_j) \\ &= f(w_0(t)x_{j,0} + w_1(t)x_{j,1} + ... + w_n(t)x_{j,n}) \end{align} $$ b. Update the weights: $$w_i(t+1)=w_i(t)+r.(d_j-y_j(t))x_{j,i}$$ for all features $0\leq i\leq n,r$ is the learning rate.

Columbia university course pdf states the update rule in a different way:

If $y'\neq y_j$ then $w_i(t+1)=w_i(t)+y_jx_{j,i}$, else leave $w$ unchanged.

I am unable to get how these two update rules are same? That is if $y_jx_{j,i}$ is same as $r.(d_j-y_j(t))x_{j,i}$?

PS: I have changed the variables in columbia university pdf to match those in wikipdia.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.