1
$\begingroup$

I have a question regarding variable following or not a random distribution. I selected 4 features negatively correlated to the label (Fraud/No Fraud). The notebook I'm taking the inspiration from plotted the distribution of these feature regarding the label. What came out is that my feature 1 (Fraud only) is following a Normal Distribution.

Here are my questions :

  1. Why is it important to know if my feature is following a Normal Distribution ? -> My guess : some models need it for faster convergence or better results
  2. Is there any interest to visualize my features as Non Fraud vs Fraud and compare the distributions ?
  3. If my features are not following a Normal Distribution but are scaled, should I still force them to a Gaussian like shape ?

Thank you very much !

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$
  1. It completely depends on the type of model. Some models need to represent the features with parameters: for example Naive Bayes with numerical features needs to have a way to calculate the probability based on the value, and the most common case is to assume that the features follow a normal distribution. On the other hand whether a feature is normally distributed or not doesn't matter at all for Decision Trees.
  2. Yes, it can be very informative in order to know whether this feature is a good indicator or not: the more different the distributions, the more easily the algorithm can distinguish the classes using this feature.
  3. No, don't change the distribution of a feature (unless you have a specific reason to do so, e.g. based on expert-knowledge for this particular data). Any way you would do that would certainly alter the overall distribution of the data and/or the way the features are related within an instance, so the model would not learn from the true distribution and therefore its predictions on real data would likely go wrong.
Improve this answer
$\endgroup$
5
  • $\begingroup$ wow thank you so much ! It is now very clear to me. This second point is crucial for me, I feel like I've learned something so powerful :). Thank you very much for your time Erwan ! $\endgroup$
    – Valentin
    Feb 6, 2021 at 10:07
  • $\begingroup$ @Erwan could you reference a source to 3.? I was always under the assumption that this should be down, when possible. $\endgroup$
    – Michael Paris
    Sep 22, 2022 at 13:02
  • $\begingroup$ @MichaelParis sorry I don't understand: what should be down? $\endgroup$
    – Erwan
    Sep 23, 2022 at 9:23
  • $\begingroup$ @Erwan, I meant to say that "this should be done, when possible". Still could you provide a reference to 3.? $\endgroup$
    – Michael Paris
    Sep 23, 2022 at 9:46
  • $\begingroup$ @MichaelParis oh ok. No, I don't have any reference, but my answer was related to the fact that OP has 4 features, so forcing an individual feature to follow a normal distribution would distort the relation this feature has with the other features and/or the target. This is different from scaling a feature, which doesn't break the relation with other variables. $\endgroup$
    – Erwan
    Sep 23, 2022 at 10:58

Not the answer you're looking for? Browse other questions tagged or ask your own question.