Interpret results from a lightFM factorization machines

Question

I built a recommendation model on a user-item transactional dataset where each transaction is represented by 1:

model = LightFM(learning_rate=0.05, loss='warp')

Here are the results of this recommendation model:

Train precision at k=3:  0.115301
Test precision at k=3:  0.0209936

Train auc score:  0.978294
Test auc score : 0.810757

Train recall at k=3:  0.238312330233
Test recall at k=3:  0.0621618086561

Can anyone help me interpret this result? How is it that I am getting such good AUC score and such bad precision/recall? The precision/recall gets even worse for 'bpr' Bayesian personalized ranking.

Prediction task:

users = [0]
items = np.array([13433, 13434, 13435, 13436, 13437, 13438, 13439, 13440])
model.predict(users, item)

Result from the prediction task:

array([-1.45337546, -1.39952552, -1.44265926, -0.83335167, -0.52803332,
   -1.06252205, -1.45194077, -0.68543684])

How do I interpret the prediction scores?

Thanks

Answer 1

The prediction scores are only used for ranking. The scores themselves do not provide more insight than that.

Answer 2

Precision@K measures the proportion of positive items among the K highest-ranked items while AUC measures the quality of the overall ranking.

For more details, you should check out this answer from Maciej Kula.