Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I was reading this article where I came across the following statement in the context of "Why do we use sigmoid activation function in Neural Nets?":
The assumption of a dependent variable to follow a sigmoid function inherently assumes a Gaussian distribution for the independent variable which is a general distribution we see for a lot of randomly occurring events and this is a good generic distribution to start with.
Could someone elaborate on this relationship between the two?
Yes sure, the core idea is that the sigmoid function approximate very nicely the cumulative density function of a Gaussian distribution. Therefore, if the input of a neuron is a continuous Gaussian distribution, the probability of the output being inferior to a certain value will follow (almost) the shape of the sigmoid function.
A detailed explanation can be found p16 of the following article:
