0
$\begingroup$

I am new to machine learning. Does it make sense that my model works when I do both Standardise and Normalise? What does this say about my data?

Or do I do have to select one or the other? My goal is to build a binary classifier which analyses CNNs and GIST features?

$\endgroup$
2
$\begingroup$

No, actually it doesn’t make sense to apply two scaling methods to your data simultaneously. Generally, I would suggest using standardization because it is more robust to outlier samples compared to normalization. Having removed outlier samples from your dataset, you can safely use normalization instead.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Apologies for the late comment, I understand from your answer that standardisation is a better fit but my problem is that when I only perform standardisation I do not get accurate results. But when I do both, normalisation and standardisation, I get the '-1' as the majority which is what I expect. Do you see any way I can fix this? $\endgroup$ – Oman Apr 13 '19 at 11:09
1
$\begingroup$

Standardization is actually a type of normalization (see below for an explanation of normalization). Generally, the goal of normalization is to scale a distribution in some way so that they may be compared to one another in the future (see below for more detailed explanation of normalization.) However -- The goal of standardization is to specifically, produce a distribution that has mean 0 and a standard deviation of 1. To obtain the standardized distribution you subtract every sample by the mean of the population then divide by the standard deviation of the population: $${\frac {X-\mu }{\sigma }}$$ Frequently in industry (as opposed to academia), people refer to normalization as max-min scaling where the distribution is scaled from 0 to 1. This is probably how you are thinking normalization is defined. However, if you think of max-min scaling, and standardization as both types of normalization you can maybe imagine why you wouldn't want to do both. The goal is to produce distributions that can be compared to one another. One method is enough to do so. Therefore the question is really what is the best way to perform normalization for my specific problem? Generally (and including CNNs as you ask about), its best to go with standardization.

In statistics and applications of statistics, normalization can have a range of meanings.In the simplest cases, normalization of ratings means adjusting values measured on different scales to a notionally common scale, often prior to averaging. In more complicated cases, normalization may refer to more sophisticated adjustments where the intention is to bring the entire probability distributions of adjusted values into alignment. In the case of normalization of scores in educational assessment, there may be an intention to align distributions to a normal distribution. (From Wikipedia)

|improve this answer|||||
$\endgroup$
  • $\begingroup$ What would it mean that if I perform both I get similar results to what I want, i.e '-1' as my majority class in the binary classifier and if I perform either I get the other class as majority by a big margin (inaccurate), what do you think this means and how do you propose I can fix it? $\endgroup$ – Oman Apr 13 '19 at 11:12
  • $\begingroup$ @Oman What do you mean when you say "When I perform both I get similar results to what I want"? Specifically, what do you mean by results? Are you talking about evaluating your predictions after building the model? $\endgroup$ – fractalnature Apr 13 '19 at 21:33
  • $\begingroup$ Thank you very much for your response. Yes, exactly I am aware of what the results should look like i.e. what the test output proportions are, 7:3, '-1' being the majority class. When I perform both normalisation and standardisation, I get '-1' being the majority class. But when I ONLY perform standardisation I get the other class as the majority. $\endgroup$ – Oman Apr 14 '19 at 11:30
  • $\begingroup$ What metric are you using to evaluate performance of your model? $\endgroup$ – fractalnature Apr 14 '19 at 16:34
  • $\begingroup$ I am predicting my test data and printing out the predictions, I have the test proportions that I am expecting, "-1" being the majority class with a 7:3 ratio. My accuracy of the model is around 72%, but I think as I am aware of the output of the final prediction that metric would be accurate. $\endgroup$ – Oman Apr 15 '19 at 15:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.