3
$\begingroup$

What happens if my data/feature is not normal? Can I still use machine learning algorithms to utilize such data for predictions?

I noticed in many data sciences courses, there is always a strong assumption of using a normal/Gaussian data. I have always wonder why this is so, and most people would say that due to central limit theorem, the data is always assumed to be normal.

  • However, what if the data that I am dealing with is not normally distributed?
  • Should I perform log/ exponential transformation on the data for the sake of getting normal distributed data?
  • Why is Gaussian data always best suited?
$\endgroup$
6
$\begingroup$

There are models that do not make assumption that the underlying data distribution is a normal distribution.

For example, support vector machine just cares about the boundaries of the separating hyperplane and do not assume the exact shape of the distributions. Decision tree models also do not make such assumption.

Gaussian distribution is popular and it has been analyzed frequently for its simplicity but there are other models as well.

If you know the distributions that you believed your data can follow, you can also build your own model, for example by maximizing the likelihood or posterior distributrion.

If you are able to convert your data to normal distribution and use a model that you are familiar with, you can try it and see how does it perform.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for your response. I would like to further clarify because I still do not understand why it is necessary to have a Gaussian distribution. Please forgive me for my lack of fundamentals. If I use a data of unknown distribution, and I used a machine learning algorithm like ridge regression or any other regression, how does it affect my predicted result? Or how does it affect the algorithm when an unknown distribution data is applied? Thank you once again. $\endgroup$ – Newbie01 Nov 4 '17 at 23:05
  • $\begingroup$ If Gaussian distribution assumption holds, your textbook result holds for sure. If the condition doesn't hold, it doesn't mean the result is going to be bad. It could be good or bad. We shouldn't reject the idea of to use it just because the assumption doesn't hold. $\endgroup$ – Siong Thye Goh Nov 5 '17 at 5:08
1
$\begingroup$

Gaussian models are often used (and maybe sometimes over-used) because of their mathematical convenience (many statistical models can be found as built-in functions, when based on the Gaussian distribution, in some libraries such as mixture models, hidden Markov models,...). Also, when one has no idea about what distribution could best model the data, given the relationship of the Gaussian distribution with the Central-Limit Theorem, it is often a reasonable assumption to make.

However, if say, your goal is to generate new data similar to some training data and that you know that the data you would like to generate should be strictly positive, then a Gaussian assumption may not be the best one. Indeed, generating data from a Gaussian-based model gives no guarantee that these data will be positive (the Gaussian has an infinite support). Then one could think of basing the model on some distribution that insure generated data to be positive such as the Beta or the Dirichlet distribution.

However, it is always a good start to take the Gaussian as an assumption, get some results and if they are not good enough, try other assumptions and compare. This can enhance the accuracy but can also require a big amount of work as most classical machine learning algorithms are often not implemented in main libraries for assumptions other than the Gaussian.

To summarize:

  • However, what if the data that I am dealing with is not normally distributed?

Then you should see an improvement of your results when using more appropriate distributions.

  • Should I perform log/ exponential transformation on the data for the sake of getting normal distributed data?

It can indeed be worth it (and little work) to transform the data and see the impact on the results.

  • Why is Gaussian data always best suited?

It is not. As a proof, you can look at several publications about mixture models that compare for different data sets how Dirichlet, Gaussian, Beta mixtures models (and other) behave.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.