I'm reading Neural Networks and Deep Learning and running into trouble with the math. One of the exercises says:
Write out $a'=\sigma (wa + b)$ in component form, and verify that it gives the same result as the rule $$\frac{1}{1 + \exp(-\sum_{j}w_jx_j - b)}$$for computing the output of a sigmoid neuron.
Not even sure where to start here. Can anyone help me out? I'd really appreciate a detailed explanation.
In your first equation w is weight matrix $(w_1 + w_2 + \ldots + w_j)$ and a is the input vector $(x_1 + x_2 + \ldots + x_j)$. Writing wa in component form means taking the dot product of these two vectors, which is $\sum_{j}w_ja_j$. Now, you just apply the sigmoid function to get your final answer:$$\sigma (z) = \frac{1}{1 + \exp(-z)}$$ $$\Rightarrow\sigma (\textbf{wa} + b) = \frac{1}{1 + \exp(-\sum_{j}w_jx_j - b)}$$
