I have customer buying data with each row specifying an item bought by customer. The problem is that even if at the same time customer buys five items then there are five different rows for it and as a result the total number of rows in data have gone too much to train. what can i do to reduce the size of data so that i can train it effectively.Just to give the context of the problem, i want to recommend products to the customers based on their buying data.

Dataset size: (7981262, 16)

Data description:

Variable                                           Description
customerID                                    unique customer ID
DOB                                         date of birth of customer
Gender                                               gender
State                                          customer's state
PinCode                                pincode of area where customer lives
transactionDate                                  date of transaction
store_code                                 unique code of store
store_description                             description of store
till_no                                       counter no. in the store
transaction_number_by_till                 unique transaction number by counter, 
transactionDate, store_code
promo_code                      if promotional code (offer) used in the transaction
promo_description                            description of the offer
product_code                               unique code of the product purchased
product_description                       description of the product purchased
sale_price_after_promo                 sale price of the product after applying 
discountUsed                  after promo, customer used this discount(s) on transaction
  • 1
    $\begingroup$ What exactly do you want your model to predict? $\endgroup$
    – Paul
    Nov 14, 2017 at 16:55
  • 1
    $\begingroup$ based on the customer buying data , i want to recommend products to the customers. $\endgroup$ Nov 14, 2017 at 17:28
  • $\begingroup$ I think I would try to build another DataFrame with one customer by row. Then I would add the full catalog of items as features and count on each row how many times each item as been bought. $\endgroup$
    – AlexMdg
    Jun 10, 2018 at 0:54

3 Answers 3


If your total dataset size is too much, take a random sample of n%. I suggest taking a sample of n% of users, then using all rows for each of those sampled users. If you have timestamps, I further suggest making your train/test split based on time. I.e., for a user with 10 purchased items, train on her first 8 items and try to predict her last 2 items.

  • $\begingroup$ That's not the kind of model this person is looking for. $\endgroup$
    – Paul
    Nov 14, 2017 at 18:27
  • $\begingroup$ Would you elaborate? I think there are quite a few possible models for this style of data, and the original poster was just asking about reducing dataset size. $\endgroup$ Nov 14, 2017 at 18:47
  • $\begingroup$ The clarification is asking about a recommendation engine, so it's not the same structure as what you give. $\endgroup$
    – Paul
    Nov 14, 2017 at 19:19

I think you might have heard something called as Association Mining, even if you don't know please go through the link.

So the reason for me to suggest this is, before giving the raw transactions to the model. We need to convert them into a baskets i.e., if a customer bought 3 items then all of them will fall under the same basket using Customer ID(any unique identifier).

It can be done by a command,

transDat <- as (myDataFrame, "transactions") # convert to 'transactions' class

By this you will get the baskets that is nothing but your desired outcome. Once that is done you can directly apply Association Mining to get the desired recommendations and select the Best Recommendations.

Do let me know if you have anymore doubts.


You can check various Recommender System solutions on Github or like which use Item based filtering or user based filtering which will take care of this kind of dataset.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.