10
$\begingroup$

Its very common in recommender that we have user product data which have label as an e.g. "click". In order to learn the model, I need click and no-click data.

Simplest approach to generate is to take user-products pairs which are not found in click data. However, that may be misleading. Example:

user1, product1 (click) user2, product2 (click) user2, product3 (click) user3, product2 (click)

I can take user1 with all products except product1 and label them as "no_click" and so on. But this may not be true. Maybe user1 would have clicked product2 if he was shown product2. But just because he was shown other sets of products - he had no opportunity to decide to click/no-click product2.

So how to address unary data problem?

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ I think you answered your own question. You should be recording a concept of an impression, or show. If you showed a product and their was no click, then this is what you are looking for. $\endgroup$
    – user13684
    Nov 17 '15 at 21:41
  • $\begingroup$ But actually this is not available in recorded data. Thats what I mentioned. Data only contain what user-product pair has click label. What was shown and what was clicked is not recorded. $\endgroup$
    – p.paliwal
    Nov 18 '15 at 10:00
  • $\begingroup$ Also, even if let say - user1 was shown prod1, prod2,prod3 (and he clicked prod1) - Then user1 with prod2 and prod3 will have no-click label. But what about rest of products (prod4,prod5,...). Just because they were not shown, user had no opportunity to decide click/no-click. This does not say whether user would be interested in not-shown products - so labeling all other combination as no-click may not be true in real. This is what I also explained in question. $\endgroup$
    – p.paliwal
    Nov 18 '15 at 10:03
6
$\begingroup$

So, there are two problems.

  1. Recording impressions (shows)
  2. How to deal with non-impressions

For (1) you should be recording this information. If it is not currently being recorded you should start recording this information. Given that you do not have this information you want to provide recommendations. Fortunately, with just click data you can still create a utility matrix, see 9.1.1.

You could then use user- or item-based collaborative filtering as described in the paper. This is basically an exercise in populating the utility matrix and trying to find "scores" for unclicked items. Your recommendation would be an unclicked item with the highest score.

For (2) you will still make recommendations on unclicked items. So, that alone is not an issue. You will want to optimize your impressions however. You also cannot have full knowledge where a user can see all possible options. You need to record impressions and understand a number of things.

  • show rate of an item
  • click rate of an item
  • how to incorporate new items
  • how to optimize which items to show

This is a huge topic and basically this is the problem domain of online advertising. However, a recommendation engine tries to find items of interest in the long tail, which is a bit different from ad optimization. This is a feedback loop to evaluate your recommendation. A/B tests are common. You will want to test click rates and recommendation errors between your current system and new system.

Also see here and here.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.