1
$\begingroup$

I am working on a denoising autoencoder problem with noisy and clean signals. Before I pass the signals to my model I want to apply min-max normalization and am unsure of the correct way to apply this.

The model will see the noisy signal as the input and the output/reference signal as the clean signal (denoising autoencoders are a type of unsupervised learning where concepts of features and labels perhaps don't apply in the original sense).

The current way I am applying scaling is by fitting and transforming the noisy and clean signals separately before fitting into the model - is this the correct strategy?

from sklearn.preprocessing import MinMaxScaler

scaler_noisy = MinMaxScaler()
scaler_clean = MinMaxScaler() 
X_noisy_train = scaler_noisy.fit_transform(X_noisy_train)
X_clean_train = scaler_clean.fit_transform(X_clean_train)
$\endgroup$
0
$\begingroup$

No, this is not the correct strategy. If the transformation you apply takes any parameters, in this case the minimal and maximal values, you should first do it on the training set and then apply it to the test set to avoid data leakage. This would not matter in case of something like a log transformation, where it does not change the outcome of the transformation. Your code should look more like this:

from sklearn.preprocessing import MinMaxScaler
from sklearn.model_selection import train_test_split

noisy_train, noisy_test, clean_train, clean_test = train_test_split(noisy, clean)

scaler_noisy = MinMaxScaler()
scaler_clean = MinMaxScaler() 

noisy_train = scaler_noisy.fit_transform(noisy_train)
noisy_test = scaler_noisy.transform(noisy_test)
clean_train = scaler_clean.fit_transform(clean_train)
clean_test = scaler_noisy.transform(clean_test)

or you could just put the scaler inside a sklean pipiline. Although you would than have to use a TransformedTargetRegressor to transform the Y as well.

I don't see this as an example of unsupervised learning. In unsupervised learning you don't have any output associated with your data points. In this case your output/labels is the clean noise.

$\endgroup$
3
  • $\begingroup$ thanks for answer this makes sense. Around the data leakage could you please elaborate, what if I was to fit_transform on the test set also what would be incorrect about this and what would that cause? I know the standard is usually fit transform on train and then transform on test but just trying to wrap my head around this intuitively. Also with regards to the type of learning - autoencoders are usually seen as a type of unsupervised learning hence the statement. $\endgroup$
    – Ossz
    Oct 11 at 15:56
  • $\begingroup$ Data leakage happens when you ''leak'' information from the training set to the test set. This results in an optimistic and unreliable evaluation of the model. The idea of train and test sets is simple. You do all the preprocessing and train the model on the train set and then see how it will perform on unseen data with the test set. If you do some preprocessing based on the test set then it is not quite ''unseen'' for the model anymore. $\endgroup$
    – Mateusz
    Oct 12 at 9:05
  • $\begingroup$ I would recommend for you to read on data leakage either on this forum or on machinelearningmastery, or medium, since there are articles and discussions much better than my short answer. As to the unsupervised part, I do remember that autoencoders learn the latent representation of data in an unsupervised manner, but from your description of the problem it looks to me like there are labels to your data, which would make this supervised learning. I don't know the specifics of your problem so I might be wrong, but that is what I got from your description. $\endgroup$
    – Mateusz
    Oct 12 at 9:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.