2
$\begingroup$

I am trying build a simple binary classifier (some tree based algorithm for now) and my training data will have features aggregated at the user level. So I'll have a unique records of each user. These aggregated features are like "number of logged in sessions", "number of times profile button was clicked" etc - essentially these are website browse behavior features.

What I am trying to predict is if someone would be interested in subscribing or not. Some users might subscribe immediately after opening an account, some might do after a few days and some may not at all. My labels will be 1 (subscribed) and 0 (not subscribed).

Customers can only subscribe after logging in. So in my dataset I'll have users whose login counts range from 1-N. Hence my aggregation for features will also have wide range of values, because users that have logged in say only once will be smaller feature values than users who have logged in multiple times.

My problem is twofold:

  • Label generation - Should I only select users who have say at-least 3 logged in sessions to assign labels of subscribed on not ? I do this because users who have only one session and have not subscribed will get label as 0 (not subscribed). I don't think I should assign them as label 0 as I don't think I have enough data to correctly conclude that label 0 is apt for them.
  • Say I select users who have at-least 5 logged in sessions and generate aggregate features. I feel my model wont be trained accurately if I have variation in features because of varying number of logged in sessions. ( e.g user A has 3 sessions hence aggregate feature values will be small compared to user B who has say 10 logged in sessions). May be I should level the field by aggregating data from only the fist 3 logged in sessions for each user and see if they subscribed or not in future.

Am I thinking about this correctly ?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

The goal is to predict if a user will subscribe in the future. By definition you cannot have labelled data now about what people will do in the future. However you can phrase the problem like this: given a set of users at time $t$, predict whether they will subscribe by time $t+u$, where $u$ is for instance 1 year, 1 month or whatever duration fits your data.

Under this definition you can use any past data you have: for every unsubscribed user at time $t$ (e.g. one year ago), label them as "will subscribe" or "won't subscribe" using the data that you have now about them. You can even collect data at different points $t$ in time and calculate for each point $t$ who among the users will be subscribed within a year (i.e. the same user could potentially be used several times as an instance).

Naturally for every time $t$ you should only use users who were not subscribed at this time $t$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you Erwan ! Two questions: * at time t we'll have users who have logged in many times and uses who probably have logged in only a few times. So I feel the aggregate features wont be at the same level. I'll have a decent data on some users and not enough data on others. So do you feel we should say only include users who have X number of logged in sessions at time t while training ? * "You can even collect data at different points 𝑡 in time" : I'll have to retrain the model right for every time I change t ? $\endgroup$
    – cryp
    Aug 8, 2021 at 12:20
  • 1
    $\begingroup$ @cryp it could make sense to discard users for whom there's not enough data indeed, but this means that you can't predict for any such users either. The proportion of these cases is probably an important information for this decision. Second question: not necessarily, training a different model for every time t means making the model itself time-dependent. I would say that this choice depends if the behaviour of the users is likely to change a lot across time: if not (or not much), then the model can use data from different times $t$, as long as the features are always relative to $t$. $\endgroup$
    – Erwan
    Aug 8, 2021 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.