0
$\begingroup$

My question is very basic. I am starting with ML and am working on the perceptron algorithm. I successfully computed the weights for this input data:

X = [[0.8, 0.1], [0.7, 0.2], [0.9, 0.3], [0.3, 0.8], [0.1, 0.7], [0.1, 0.9]]
Y = [-1, -1, -1, 1, 1, 1]

Output_weights = [-0.7, 0.5]

But I didn't take bias into account, i.e. I assumed the discriminator line goes through the origin. Now, let's say I add another point into my training set:

new_X = [4,4]
new_Y = [-1]

How do I proceed if I want to compute the bias as well? In the first iteration for example, I'd set default weights to $[0,0]$, so I find the first point that is incorrectly classified.

Without bias, it is easy. I compute the dot product

0.8*0 + 0.1*0 = 0

should be $-1$, so it is incorrectly classified. I update the weights to:

[-0.8,-0.1]

However, taking bias into account, I get:

0.8*0 + 0.1*0 + bias 

Now, how do I update the weights and the bias? What is the procedure?

I have searched several tutorials like this or this but didn't find an answer. A link to some resource would help, too.

$\endgroup$
2
$\begingroup$

Rather than viewing your data as

X = [[0.8, 0.1], [0.7, 0.2], [0.9, 0.3], [0.3, 0.8], [0.1, 0.7], [0.1, 0.9]]
Y = [-1, -1, -1, 1, 1, 1]

You could have treat the problem as

X = [[1, 0.8, 0.1], [1, 0.7, 0.2], [1, 0.9, 0.3], [1, 0.3, 0.8], [1, 0.1, 0.7], [1, 0.1, 0.9]]
Y = [-1, -1, -1, 1, 1, 1]

That is to append a $1$ in every single entry of $X$. The weight corresponding to $1$ would be the bias.

That is $Y=Ax+b$ can be written as $Y=[A, e]\begin{bmatrix}x \\b\end{bmatrix}$ where $e$ is the all one vector.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Great, but what happens if I actually need to obtain the value of b? Is it just then -1 * np.dot(A, x) - Y? $\endgroup$ – Eduardo Marcano Jun 11 at 18:50
  • $\begingroup$ You solve for $x$ and $b$ simultaneously using the perceptron algorithm in order to get your $b$. $\endgroup$ – Siong Thye Goh Jun 12 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.