0
$\begingroup$

I built a recommendation model on a user-item transactional dataset where each transaction is represented by 1:

model = LightFM(learning_rate=0.05, loss='warp')

Here are the results of this recommendation model:

Train precision at k=3:  0.115301
Test precision at k=3:  0.0209936

Train auc score:  0.978294
Test auc score : 0.810757

Train recall at k=3:  0.238312330233
Test recall at k=3:  0.0621618086561

Can anyone help me interpret this result? How is it that I am getting such good AUC score and such bad precision/recall? The precision/recall gets even worse for 'bpr' Bayesian personalized ranking.

Prediction task:

users = [0]
items = np.array([13433, 13434, 13435, 13436, 13437, 13438, 13439, 13440])
model.predict(users, item)

Result from the prediction task:

array([-1.45337546, -1.39952552, -1.44265926, -0.83335167, -0.52803332,
   -1.06252205, -1.45194077, -0.68543684])

How do I interpret the prediction scores?

Thanks

$\endgroup$

2 Answers 2

1
$\begingroup$

The prediction scores are only used for ranking. The scores themselves do not provide more insight than that.

$\endgroup$
1
$\begingroup$

Precision@K measures the proportion of positive items among the K highest-ranked items while AUC measures the quality of the overall ranking.

For more details, you should check out this answer from Maciej Kula.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.