Data leakage and predictive models: should we use past predictions as a feature?

Question

I want to develop a Random Forest Classifier model to predict whether or not a customer will convert 7 days from today. The model is re-trained once a week and makes predictions for the following week. The features I use were created using 2 years worth customer behavior data.

Since I'm running this model and generating predictions once a week, I have been storing the conversion predictions for every customer on a weekly basis. Furthermore, I will know if my previous predictions were correct or not.

So let's say I'm training the model again today for the coming week and I want to include following features:

last_wk_predictions = Probability of Purchase predicted last week for each customer (between 0 and 1)
did_convert_last_wk = did they convert last week? (0 or 1)

Is there a specific name for this approach? Is it considered data leakage to include past predictions?

Answer 1

It depends on the business problem you are solving. You might see data leakage if your initial models were not so good. For the weekly problem it will be good to get the real ground truth (even a fraction of the week's data will be good) and add it for reevaluation of the model and possible retraining. Now if you are thinking that past purchase will make the customer return back (yes if the products are good and quickly consumed making the customer repeatedly want it again) then yes adding every week's ground truth will definitely greatly increase reduce the misclassification for returning customers for same products. But as I said in my first sentence if you only include predictions of past week into this week the model might go wrong since first iterations errors will start influencing the later iterations and the model will move away from good prediction as time passes by. If you include predictions of past week into this week and the customer's returning behaviour has got nothing or very less to do with the current behavior (like non-chronic patient visiting the hospital for Outpatient visits for minor ailments) then you are introducing data leakage which will give wrong predictions as time passes by.