I have a question regarding variable following or not a random distribution. I selected 4 features negatively correlated to the label (Fraud/No Fraud). The notebook I'm taking the inspiration from plotted the distribution of these feature regarding the label. What came out is that my feature 1 (Fraud only) is following a Normal Distribution.

Here are my questions :

  1. Why is it important to know if my feature is following a Normal Distribution ? -> My guess : some models need it for faster convergence or better results
  2. Is there any interest to visualize my features as Non Fraud vs Fraud and compare the distributions ?
  3. If my features are not following a Normal Distribution but are scaled, should I still force them to a Gaussian like shape ?

Thank you very much !


1 Answer 1

  1. It completely depends on the type of model. Some models need to represent the features with parameters: for example Naive Bayes with numerical features needs to have a way to calculate the probability based on the value, and the most common case is to assume that the features follow a normal distribution. On the other hand whether a feature is normally distributed or not doesn't matter at all for Decision Trees.
  2. Yes, it can be very informative in order to know whether this feature is a good indicator or not: the more different the distributions, the more easily the algorithm can distinguish the classes using this feature.
  3. No, don't change the distribution of a feature (unless you have a specific reason to do so, e.g. based on expert-knowledge for this particular data). Any way you would do that would certainly alter the overall distribution of the data and/or the way the features are related within an instance, so the model would not learn from the true distribution and therefore its predictions on real data would likely go wrong.
  • $\begingroup$ wow thank you so much ! It is now very clear to me. This second point is crucial for me, I feel like I've learned something so powerful :). Thank you very much for your time Erwan ! $\endgroup$
    – Valentin
    Feb 6, 2021 at 10:07
  • $\begingroup$ @Erwan could you reference a source to 3.? I was always under the assumption that this should be down, when possible. $\endgroup$ Sep 22, 2022 at 13:02
  • $\begingroup$ @MichaelParis sorry I don't understand: what should be down? $\endgroup$
    – Erwan
    Sep 23, 2022 at 9:23
  • $\begingroup$ @Erwan, I meant to say that "this should be done, when possible". Still could you provide a reference to 3.? $\endgroup$ Sep 23, 2022 at 9:46
  • $\begingroup$ @MichaelParis oh ok. No, I don't have any reference, but my answer was related to the fact that OP has 4 features, so forcing an individual feature to follow a normal distribution would distort the relation this feature has with the other features and/or the target. This is different from scaling a feature, which doesn't break the relation with other variables. $\endgroup$
    – Erwan
    Sep 23, 2022 at 10:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.