I have been reading about probability distributions lately and saw that the Normal Distribution is of great importance. A couple of the articles stated that it is advised for the data to follow normal distribution. Why is that so? What upper hand do I have if my data follows normal distribution and not any other distribution.
This is an interesting question. So sorry for a long winded answer. The tl:dr; is it is a mix of some real applicability, theoretical basis, historical baggage (due to limited compute power) and obsession for analytically tractable models (instead of simulation/computational models). We should be very careful and discerning while using it in real problems.
The importance of normal distribution comes from the following facts/observations,
- Many naturally occurring phenomenon seem to follow normal distribution when sample size is large (more on this below).
- In Bayesian statistics, if you assume a Normal distribution prior on parameters, then posterior distribution is also normal. This makes computations easier.
- Somewhat related, central limit theorem tells us that average of a samples from any distribution (no fat tails) follows normal distribution. So normal distribution is useful and provides theoretical basis for doing population level parameter estimates from samples (think of election predictions). But again, this assumes the underlying data comes from a distribution which is well behaved and extreme values are very unlikely.
In short, normal distribution can be thought of as a good base case, which is analytically tractable, easy to code up and also seems to be applicable to many models of nature. Somewhat broken analogy, but in Physics we consider linear second order differential equations to study many systems. Now not all systems actually are linear second order, but it is a reasonable approximation under some constraints that is easier to analyze and code up.
And over-usage of normal distribution everywhere is actually controversial.
- As we have to more computing power and access to Monte Carlo based simulation based method, we are no longer limited by using only analytically tractable distributions. We can use distributions which more accurately fit the reality.
- Normal distributions are useful for natural phenomenon (heights of students in a class) but are way inaccurate to model mostly man made systems (income of people in a town, potential swings of stock indices during panic).
- For example, many critics of probabilistic financial models observe that the underlying models use normal distribution. But real market swings are mostly fat tailed (distributions where extreme outcomes are more likely than normal distributions). If you want to go down on deeper into this, start with statistical consequences of fat tails by Nassim Nicholas Taleb. Fun fact, if you look at the wild swings in the price of GameStop stock from the r/wallstreeetbets saga, Taleb pointed out that the swings are not actually wild if you consider a fat tailed distribution.