Machine Learning Perceptron Algorithm

Question

I'm studying Machine Learning using Sebastian Raschka's book.

1. Wonder if someone could please help me to confirm if I have the following steps correct if I apply Perceptron Algorithm to Iris dataset as shown in the table below.
2. What do I set $\eta$ to?
3. What do I set $θ$ to? I randomly set it to 2.

Perceptron rule - summarized by the following steps:

1. Initialize the weights to 0 or small random numbers.

2. For each training sample $x(i)$ perform the following steps:

   • Compute the output value $\hat{y}$ .

   • Update the weights.

\begin{array}{c|c|c|} & \text{sepal_length, $X_1$ } & \text{sepal_width, $X_2$} & \text{petal_length, $X_3$} & \text{petal_width, $X_4$} & \text{species, $Y$}\\ \hline \text{Row 0} & 5 .1 & 3.5 &1.4 &0.2 & setosa \\ \hline \text{Row 1} & 4.9 & 3.0 & 1.4 & 0.2 & setosa \\\hline \text{Row 2} & 4.7 & 3.0 & 1.4 & 0.2 & setosa \\\hline \text{Row 3} & 4.6 & 3.1 & 1.5 & 0.2 & setosa \\\hline \text{Row 4} & 5.0 & 3.6 & 1.4 & 0.2 & setosa \\\hline \end{array}

set: $θ = 2$

set: $virginica = -1,$ $\>$ $setosa = 1$

let's say if $z \ge θ,$ $\>$ $θ(z) = 1$, $\>$ $θ(z) = -1$ if otherwise

Initialize weight vector: $W$ = [ $w_1$ = 0.1, $w_2$ = 0.2, $w_3$ = 0.3, $w_4$ = 0.4]

For observation row 0:

$z = w_1*x_1 + w_2*x_2 + w_3*x_3 + w_4*x_4$ $= (0.1* 5.1) + (0.2*3.5) + (0.3*1.4) + (0.4*0.2)$ $= 1.71$

Since $z = 1.71 \lt θ = 2$, $\>$ then $θ(z) = -1$

As for row 0, the perceptron algorithm incorrectly predict $virginica = -1,$, hence the weights would be adjusted.

$w_j:= w_j + \Delta w_j$

$\Delta w_j = \eta(y^{(i)} - \hat{y}^{(i)}) x_j^{(i)}$

Hence,

$\Delta w_1 = \eta(1-(-1))5.1 = 10.2\eta$

$\Delta w_2 = \eta(1-(-1))3.5 = 7.0\eta$

$\Delta w_3 = \eta(1-(-1))1.4 = 2.8\eta$

$\Delta w_3 = \eta(1-(-1))0.2= 0.4\eta$

Update weight vector: $W$ = [ $w_1 = 0.1 + 10.2\eta$, $w_2 = 0.2 + 7.0\eta$, $w_3 = 0.3 + 2.8\eta$, $w_4 = 0.4 +0.4\eta$]

Using the updated weight to Row 1 and repeat the process above.

Answer 1

OK WHAT I understood from your understanding is below:-

1.What do I set η to? Answer:**It is learning rate. As usual everyone uses **η=0.01. Learning rate is crucial because if you set bigger values when updating weights the it will reach to a local minima and never reach to global minima. Now if you set it low then it will gradually went to global minima. This iris dataset problem is so trivial that it takes no time to reach global minima but for a complex dataset you will need another factor such as decay rate. After a while, when training you will find the accuracy isn't increasing then the neural net suppose that it reached global minima now if you set decay rate to learning rate like 0.01* decay rate then learning rate will be much less and then it will TRULY converges to global minima as the gradient will be minimized in a small amount.

2.Z is the label for your dataset.You should have used one hot encoding. I think rather than writting everything here i would suggest following links. Please go to there and then pick up a paper and pen.do the math.Bang.

How Neural Network learns

And obviously check 3Blue1Brown youtube channel's playlist on Neural Net.

Welcome to Deep Learning.Best of luck.