How to most effectively utilize historical data to train churn model

Question

Suppose we have some historical data of users activity on a website and we want to build a churn prediction model (let's say we want to predict churn in a 2 month window). The usual approach, as I understand it, is to take a slice of that historical data at time $t$ and see which users churn in the time interval $(t, t + 2 \text{ month})$, so we take some features at time $t$ and train our model.

However, that way we only use a small part of our data, feeding our model only users who were active at time $t$. But what if we want to use all historical data? One way that comes to mind is take a lot of slices of our data at times $t_1, t_2, t_3, \dots$ and just merge them in one dataset, however different slices could have the a lot of same users even if we take these slices very far apart from each other. So our model could potentially learn that if a particular set of features occurs many times in our data set, then the user with these features is less likely to churn, (e.g. if we take two time slices at $t_1$ and $t_2>1$ then, if a user is presented in both of these slices, he can't churn at least at time $t_1$). So it doesn't seem to be the right way to do it...

How can I extract as much information as possible from a historical data over large period of time without spoiling the model?

Answer 1

That's correct. You will have the same user multiple times. This can introduce a small bias. I would make a few points why this is better than working with your first approach where you have less data:

First, if the same user shows up in multiple slices that means he doesn't churn or churns later than others. So you bias the model towards figuring out what high retention users look like. As you say:

So our model could potentially learn that if a particular set of features occurs many times in our data set, then the user with these features is less likely to churn

and this is actually a good thing if you have a good mix of user retention periods.

Second, you can include the time $t$ as a feature (or their signup date etc.). This will teach your model that users that haven't churned in the past are less likely to churn in the future and reduce the importance the model puts on these other patterns you mention.

More generally, you should not only have a set of static features that don't change for the same user from $t$ to $t+1$. You should also have set of features that is dependent on $t$, e.g. "the number of logins in the last 7 days". That way the model will see differences between the same user at time $t$ and $t+1$.