1
$\begingroup$

I have a time-varying dataset that contains some missing data. I have sensors that continuously monitor some properties at evenly-spaced intervals and I would like to impute the missing values using basic interpolation for both the training and test set. This is a time-series binary classification problem (e.g., based on the entire time-series present, classify as either 1 or 0). I am concerned that taking data from the future to interpolate the missing value is a form of data leakage.

My reason for believing it is not is primarily based on the fact that I am not doing forecasting. I am not trying to predict future values of these sensors, just impute the missing dynamic variables with its most likely values (in fact, based on my domain knowledge and some experiments simple interpolation is very accurate at predicting the true values). If I were trying to predict future sensor values (e.g., forecasting) this would certainly be data leakage, correct?

$\endgroup$
2
$\begingroup$

If you are using information from the future to impute missing data would be data leakage as you would not have this extra information when the model is in production and trying to predict future values. To prevent data leakage, make sure to only use values that are available at the date/time you want to predict. If you were to impute the missing data based on historical data you would not be leaking data since you have this data available at the moment of prediction (as the term "historical" implies).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.