update model prediction for each day closer to the event

Question

I want to predict whether people will renew their yearly subscriptions. I want to make this prediction though for each user on every day of their subscription up until the day before the subscription ends (i.e., make predictions for each user on day 1, 2, 3... 364).

A simple approach is to obtain training data by randomly sampling users at some point in their history (e.g., look at the information we knew about user 1 when they were 300 days away from renewal; look at what we knew about user 2 when they were 23 days away from renewal, etc) and then build a model with a "days away" variable.

One issue with this approach is that unless you have a ton of data, the model won't perform as well on any given "days away" (e.g., 30 days before) as a model in which all data was trained with the same days_away (all users data was limited to what we knew about them exactly 30 days before they were up for renewal).

I'm curious now about other people's thoughts about how to best train a model(s) that can update its prediction each day as the prediction event gets closer in time. Are there better ways to approach it than the approach outline above?

Answer 1

I would rephrase the problem into a business problem. I.e., What we are trying to accomplish here.

From the sound of it, it seems like a Churn prediction problem. The article outlines some ideas and potential solutions.

In simpler terms, if the goal is to prevent churn by executing the right marketing messages, etc.

1. How can we predict the risk of someone leaving? Since you have historic data, it can be a supervised learning algorithm to help you identify such risk.
2. What other features do we want to add to the model? Actions / usage data / web visitation / email open, etc. leading to the renewal days. You can normalize the data by the days prior to renewal, etc.
3. Determine what actions you want to take; it will help you determine if you want to get a risk score, or a classifier with differing level of risk, etc.

Answer 2

The usual way you would tackle this is to create a row for each user and each day of the year and then predict whether someone will cancel their subscription within a window of time, say the next 30 days. If everyone is on a fixed annual billing cycle, usually people don't cancel prematurely, so that the number of days until the next billing cycle will be important to include as a feature.

It's also important to only include information up until the day under consideration to avoid data leakage. Sometimes it is easier to use a rolling window (looking back in time) to calculate features, e.g. how many emails have been sent to this person within the last 30 days. This helps to keep the feature space consistent at each day.