Say I split my raw data into train and test sets. Should I clean them first and denoise the datasets before I start creating new features or, should I create new features for both the train and test set and then clean/denoise them?
I'm looking to create my own Transformers for use in an sklearn ML pipeline but I am unsure about the order in which to do things.
p.s. I would be performing cross-validation and want to prevent data leakage.
Not sure what kind of data you have. In case your using images, is done by image, in which case, you wouldn't have data leakage if you split before or after. But if your talking about a time-related dataset where cleaning/ denoising could consist of using moving averages of statistics from thew whole dataset, you should definitely split first to avoid that statistics from a validation/test dataset get in contact with the training dataset. As a rule of thumb, always ask yourself: when I am using this model in production, how will the data presented to me be?
A simple and more trivial example that illustrates is when you fit a scaler in your training set to use it later in the validation/test dataset, since in production you will have a scaler pre-trained and will have to use it against the new data that is presented to you.
And more specific for the case you want to use cross-validation: For each fold you should do your denoising of what is validation and training separately.
If that wasn't super helpful, could you clarify what type of data you are using and what exaclty do you intend to do as denoising?
