0
$\begingroup$

Say I split my raw data into train and test sets. Should I clean them first and denoise the datasets before I start creating new features or, should I create new features for both the train and test set and then clean/denoise them?

I'm looking to create my own Transformers for use in an sklearn ML pipeline but I am unsure about the order in which to do things.

p.s. I would be performing cross-validation and want to prevent data leakage.

$\endgroup$

1 Answer 1

1
$\begingroup$

Not sure what kind of data you have. In case your using images, is done by image, in which case, you wouldn't have data leakage if you split before or after. But if your talking about a time-related dataset where cleaning/ denoising could consist of using moving averages of statistics from thew whole dataset, you should definitely split first to avoid that statistics from a validation/test dataset get in contact with the training dataset. As a rule of thumb, always ask yourself: when I am using this model in production, how will the data presented to me be?

A simple and more trivial example that illustrates is when you fit a scaler in your training set to use it later in the validation/test dataset, since in production you will have a scaler pre-trained and will have to use it against the new data that is presented to you.

And more specific for the case you want to use cross-validation: For each fold you should do your denoising of what is validation and training separately.

If that wasn't super helpful, could you clarify what type of data you are using and what exaclty do you intend to do as denoising?

$\endgroup$
1
  • $\begingroup$ I have a nonstationary raw time-series signal. So I should separate this data into train and test. Then I can apply my sklearn Denoising Transformer (e.g. LOWESS or Kalman Filter) which will be applied to train, validation folds and test set separately. In which after I will use another Transformer to engineer stationary features from the non-stationary denoised signal and drop the non-stationary feature. I don't really know if this would be acceptable as in the case of live production I don't understand how feature engineering is done when model is deployed. $\endgroup$ Aug 10, 2022 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.