1
$\begingroup$

In all implementations of recommender systems I've seen so far, the train-test split is performed in this manner:

+------+------+--------+
| user | item | rating |
+------+------+--------+
| u1   | i1   |    2.3 |
| u2   | i2   |    5.3 |
| u1   | i4   |    1.0 |
| u3   | i5   |    1.6 |
| ...  | ...  |    ... |
+------+------+--------+

This is transformed into a rating matrix of the form:

+------+-------+-------+-------+-------+-------+-----+
| user | item1 | item2 | item3 | item4 | item5 | ... |
+------+-------+-------+-------+-------+-------+-----+
| u1   | 2.3   | 1.7   |   0.5 |   1.0 | NaN   | ... |
| u2   | NaN   | 5.3   |   1.0 |   0.2 | 4.3   | ... |
| u3   | NaN   | NaN   |   2.1 |   1.3 | 1.6   | ... |
| ...  | ...   | ...   |   ... |   ... | ...   | ... |
+------+-------+-------+-------+-------+-------+-----+

where NaN corresponds to the situation where a user has not rated that particular item.

Now, from each row (user) of the matrix, a certain percentage of the numeric (non-NaN) values are removed and set aside into a new matrix, representing the test set. The model is then trained on the initial matrix, with test samples removed, and the goal of the recommender is to fill-in the missing values, with the smallest possible error.

My question is, can the train-test split be somehow done user-wise? For example to keep a set of users separate, train the recommender on the rest of the user set and then try to predict the ratings for the new users? I know this goes a bit against the idea that "if a recommender does not know you, it cannot recommend something you like", but I am wondering if some k-NN can be done.

$\endgroup$
1
$\begingroup$

My two cents,
Evaluating results of a recommendation engine where test-set is unseen users only, will allow you doing exactly that.
Evaluating results on unseen users only.
If this is indeed the motivation(product/business wise) behind your recommendation engine I would suggest trying to tackle this problem directly.
If evaluating performance on 'unseen' users is an additional metric and not the only one I would keep with current scope of train/test splitting(which should create some unseen users in the test set).

Regarding methods on how to best recommend to unseen users, your intuition(K-nn) is one common solution.
See this for example

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.