I have been working with NNs for a while, but haven't dug too deep into this unfortunately.

By looking at the three neurons below, in each of their boxes we can see that they are really just making linear separations in the x1, x2 plane (of course not taking into account the third and upward Y dimension that make the sigmoid) and combining these into the decision boundary we see at the right.


I understand we need non-linear activation functions, but why?

  • How can combining non-linear perceptrons can make this three walled decision boundary?

  • Mathematically, can it be illustrated how we are allowed to go from a simple linear function, to a more complex non-linear function using multiple neurons?

I have looked around and articles just says that they can, but not how?


1 Answer 1


I think what you are referring to is how neural nets work as universal function approximators. Check out this link for an intuitive explanation.

  • $\begingroup$ I actually read that chapter, but since he is using a single output neuron and my problem is a classification problem I thought it couldn't be applied, but I guess it can? It is easier to understand and plot/visualize the function made by a single input single output network for regression. $\endgroup$ Commented Nov 3, 2018 at 12:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.