0
$\begingroup$

I have a multiclass problem (3 classes) that looks to predict if someone will buy a product, neutral or not. I have initial features of in-app activity data such as likes, share, bookmark, share, clicks, etc. (all of them were collected before the purchase date).

Do I still need to do Temporal Train-Validation-Test split or is a random one in scikit-learn enough? If yes, is Temporal splitting always the way to go for tabular data classification problems?

Thank you.

$\endgroup$

2 Answers 2

1
$\begingroup$

Temporal train-validation-test split is generally for time series problems or in cases where an external stimulus/intervention has led to a change in user behavior or some regime change in the dynamics of the system/the experiment. When you are certain the data points are not idependent and identically distrbuted,you cannot randomly shuffle them.

I am assuming based on reading the list of features in your question that neither of the above are true - you do not have a time component nor is there a reason to believe that some intervention has taken place. In this case a random split is reasonable.

$\endgroup$
0
$\begingroup$

Sounds like random split is the way to go, since you are using all previous data from a user to make one recommendation. You might need to fix the time dimension of in-app activity data considered for the prediction (e.g., three months), as I assume not all users will have the same activity length.

IMPORTANT:

If there are multiple data samples entries for each user (for example, multiple purchases from the same user), then I would recommend doing the split first on a PER-USER basis. At the end of the day, you want to predict user behavior, and you do that on a per-user basis. Otherwise you are corrupting your test set by including purchases by one user in both the test and the train set.

$\endgroup$
1
  • $\begingroup$ Very nice explanation. WIll update you on the progress. Thanks! $\endgroup$ Sep 2, 2023 at 9:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.