3
$\begingroup$

I know a similar post was made here, but I wanted to ask some follow up questions. I am conducting a cross-validation search to find values of a set of hyper-parameters and need to normalise the data.

If we split up the data as follows:

  1. 'Training' (call this set 'A' for now) and testing data
  2. Split the 'training' into training (call this set 'B' for now) and validation sets

what parameters should be used when normalising the datasets?

Am I correct in thinking that:

  1. We normalise dataset 'B' and then extract the means and standard deviations on it
  2. We then normalise the validation set using those parameters obtained from set 'B'
  3. Once we have used the validation set to find my hyper-parameters with cross-validation, then we normalise set 'A' and extract its parameters
  4. Use the parameters from set 'A' to normalise the testing set

Is this correct, or have I misunderstood something? I know this is basic, but I can't seem to find a straightforward answer to this anywhere?

Improve this question
$\endgroup$
1
  • $\begingroup$ Simples answe, do everything on training set apply on val and tete data $\endgroup$
    – Ashwiniku918
    Jan 16, 2022 at 8:14

1 Answer 1

Reset to default
1
$\begingroup$

I am not exactly sure what you mean by "what parameters should be used when normalizing datasets."

However, it is important to note:

Normalization is a preprocessing step that you do to some or all of the parameters of your model before constructing the model.

But in answer to your question:

You always normalize the same parameters used in both the train and the test set (otherwise how would you be able to compare the results?).

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thanks. Yes, the original question ought to have been phrased better. That does make sense. After we have done cross-validation to tune hyper parameters, do we 're-normalise' the (whole) training and testing sets? (steps 3 & 4). I am just not sure if those steps are the correct thing to do after we have found parameters to use and now want to test the model's metrics. Intuition suggests those steps 3 & 4 are correct, but I just want to double check. Thanks $\endgroup$
    – Rocky the Owl
    Dec 4, 2020 at 19:55
  • $\begingroup$ Once you get your $\hat{y}$ prediction out of the model with normalized parameters you would need to undo this if you would like to interpret it in the same context as the unnormalized values. $\endgroup$
    – Ethan
    Dec 4, 2020 at 20:11
  • 1
    $\begingroup$ Does that make more sense? $\endgroup$
    – Rocky the Owl
    Dec 4, 2020 at 20:20
  • 1
    $\begingroup$ I am with you up until after the 'Do CV'. After that stage, should we not train our model using the training and validation sets combined (thus requiring us to combine train and validation sets to form a new training set)? Then we can use this new training set to do the normalisation on the testing set? $\endgroup$
    – Rocky the Owl
    Dec 4, 2020 at 20:40
  • 1
    $\begingroup$ I know that the difference may be minimal depending on the size of the validation set, but wondering from a conceptual standpoint $\endgroup$
    – Rocky the Owl
    Dec 4, 2020 at 20:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.