Does binning a time series with pd.qcut (using quantiles) create data leakage?

Question

Let's say I want to predict whether a company will default on it's debt at some point in time (so binary classification) and one of the time series variables I'm using is the "revenue" of that company thought time. Can I binned this variable "revenue" using quantiles cut (like so => pd.qcut(df['revenue'],bins=10)) without creating a data leakage ?

I'm under the impression that I can not really do that since the quantile cut is made by knowing the entire distribution of the variable "revenue" throughout the period. That is, the bin attributed to "revenue" at any point in time in my training data will carry information about the future.

Am'I correctly assuming that this will create a data leakage for this time series prediction problem ? If so, can I safely use pd.cut instead with no quantile ?

Thank you

Answer 1

I think that indeed you may have leakage by using pd.qcut. A solution to avoid that leakage is to do it in a time-series cross-validation fashion. The idea is to derive the quantile values in a training fold, and, with those values, do the cut in its validation fold.

This is a little complex and if you want it simpler you can use pd.cut, which will have no leakage issues as long as you don't derive the cuts from the full distribution. If you use some default cuts defined by you, the risk of leakage will be very low.

Answer 2

Alright thank you, that confirm what I thought. For the pd.cut what do you mean by "not deriving the cuts from the full distribution" ? How should I pick a subset of the that distribution then ? Presumably, the same issue might arises similar to that of the pd.qcut right ? Also, is there a straight forward way of deriving quantile value from training set and then apply the cut on the validation set ? Looks like I will have to create that binned feature while creating folds.