1
$\begingroup$

I have a large dataset (20M+ rows) of user interactions which I want to use to predict the probability of a customer purchasing an item in one-, three- and six months time. However since the interactions contains a date element, I'm unsure what the best possible structure of the dataset should be to keep the valuable date information but also transform the data structure to optimize the training process of a model (leaning towards XGboost or regression model).

Below are a few samples of the datasets I have available:

Table1 (I have around 25 features similar to pruchases_past_3months):

Person_ID Date Points_balance Purchases_past_3months
P_111111 2024-01-01 10 1
P_111112 2024-01-01 15 0
P_111113 2024-01-01 30 4
P_111114 2024-01-01 20 2
P_111115 2024-01-01 15 5
P_111116 2024-01-02 50 0
P_111117 2024-01-02 40 1
... ... ...
P_111111 2024-04-28 70 2
P_111112 2024-04-28 100 1
P_111113 2024-04-28 250 5
P_111114 2024-04-28 80 2
P_111115 2024-04-28 0 6

Table2 (With purchase_date as the target variable):

Person_ID Purchase_Date Item_ID
P_111111 2024-02-12 PR_1337
P_111112 2024-03-10 PR_1337
P_111113 2024-04-05 PR_1337
P_111114 2024-03-08 PR_1337
P_111115 2024-04-10 PR_1337
P_111115 2024-04-12 PR_1337
P_111115 2024-04-18 PR_1337

I might just have a case analysis paralysis and overthinking it🥴, but would really appreciate any recommendations or tips on how to retain informational value while optimizing the dataset.

$\endgroup$
1
  • $\begingroup$ What do you mean by "purchase_date as target column"? You said before that you want to predict whether a customer buys, not in which month exactly. $\endgroup$ Commented May 12 at 20:13

1 Answer 1

1
$\begingroup$

For the date columns I would suggest

  1. Feature on given date being a bank holiday in a given country
  2. Feature on weekday, do an ordinal encoding
  3. Add seasonality indicators, which month it is, how many days it is to the closest bank Holiday, like Christmas, Easter etc.

I would not think about optimizing training process before having some baseline model. Start with a XGBRegressor and improve it with new features, feature engineering etc.

Good luck!

$\endgroup$
3
  • $\begingroup$ @MJ_VdH and have a look at this link ;) How do I ask a good question? $\endgroup$
    – dmayilyan
    Commented May 8 at 6:24
  • $\begingroup$ How does computing features from the date column help OP get a better structure in the data for learning?? $\endgroup$ Commented May 12 at 20:11
  • $\begingroup$ He asked "best possible structure of the dataset should be to keep the valuable date information but also transform the data structure to optimize the training process" and I suggested transformations and creation of new features to utilize that column. He asked how to utilize column date and I gave my opinion. $\endgroup$
    – dmayilyan
    Commented May 13 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.