Which is the loss function used for validating a CF Recommender System?

Question

I am developing (from scratch) a memory-based CF Recommender System based on movielens dataset.

My CF RS uses a URM (User Rating Matrix) where r_ij contains the rating the user i gave to movie j (or missing).

I am given a test set. Moreover I am using the k-fold cross validation for validating my model: the training set is split into training (model building set) and validation set (the fold).

Is it correct to use the MAE (Mean Absolute Error) for calculating the validation loss on the generic fold?

I am also using the MAE for calculating the accuracy of the previous selected model on the test set.

Answer 1

MAE (Mean Absolute Error) can be used to evaluate performance on hold-out data. The caveat is that MovieLens uses star ratings (1, 2, 3, 4, 5) which are measured on an ordinal scale. MAE is more commonly used to evaluate performance continuous data.