0
$\begingroup$

I was revisiting neural network basics from this post. The perceptron follows below equation:

$$\begin{align} y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i \geq \theta \\ & = 0 & \text{if } \sum_{i=1}^n w_i\times x_i \lt \theta \\ \end{align} $$

Absorbing threshold value $\theta$ to left hand side, we get:

$$\begin{align} y & = 1 & \text{if } \sum_{i=1}^n w_i\times x_i - \theta \geq 0 \\ & = 0 & \text{if } \sum_{i=1}^n w_i\times x_i - \theta \lt 0 \\ \end{align} $$

Further including $\theta$ in weights and inputs, i.e. $x_0=1$ and $w_0=-\theta$ and starting sum from $i=0$ instead of $i=1$, we get

$$\begin{align} y & = 1 & \text{if } \sum_{\color{red}{i=0}}^n w_i\times x_i \geq 0 \\ & = 0 & \text{if } \sum_{\color{red}{i=0}}^n w_i\times x_i \lt 0 \\ \end{align} $$

I knew this convention of including bias as a first value in weight vector, but it is first time I am finding "threshold value $\theta$" getting moved in as "bias weight $w_0$". Is this what always / normally happens? (that is, does all online articles and book mean the same: including threshold as bias. I never knew bias had any connection with threshold value and was always felt that it is independently learnt during training)

$\endgroup$
1

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy