How to train-test split and cross validate in Surprise?

I wrote the following code below which works:

from surprise.model_selection import cross_validate

cross_validate(algo,dataset,measures=['RMSE', 'MAE'],cv=5, verbose=False, n_jobs=-1)


However when I do this: (notice the trainset is passed here in cross_validate instead of whole dataset)

from surprise.model_selection import train_test_split
trainset, testset = train_test_split(dataset, test_size=test_size)
cross_validate(algo, trainset, measures=['RMSE', 'MAE'],cv=5, verbose=False, n_jobs=-1)


It gives the following error:

AttributeError: 'Trainset' object has no attribute 'raw_ratings'


I looked it up and Surprise documentation says that Trainset objects are not the same as dataset objects, which makes sense.

However, the documentation does not say how to convert the trainset to dataset.

My question is: 1. Is it possible to convert Surprise Trainset to surprise Dataset? 2. If not, what is the correct way to train-test split the whole dataset and cross-validate?

EDIT: It seems I misunderstood the task at first, so here's my correction. Hope it works this time

It seems like what you're trying to do is similar to what is in the documentation under examples/split_data_for_unbiased_estimation.py (or this github issue which seems to be exactly what you want)

The code manually splits the dataset into two without using any sort of function call. Then sets the internals of the data variable to be only the train split.

import random

from surprise import SVD
from surprise import Dataset
from surprise import accuracy
from surprise import GridSearch

raw_ratings = data.raw_ratings

# shuffle ratings if you want
random.shuffle(raw_ratings)

# 90% trainset, 10% testset
threshold = int(.9 * len(raw_ratings))
trainset_raw_ratings = raw_ratings[:threshold]
test_raw_ratings = raw_ratings[threshold:]

data.raw_ratings = trainset_raw_ratings  # data is now your trainset
data.split(n_folds=3)

# Select your best algo with grid search. Verbosity is buggy, I'll fix it.
print('GRID SEARCH...')
param_grid = {'n_epochs': [5, 10], 'lr_all': [0.002, 0.005]}
grid_search = GridSearch(SVD, param_grid, measures=['RMSE'], verbose=0)
grid_search.evaluate(data)

algo = grid_search.best_estimator['RMSE']

# retrain on the whole train set
trainset = data.build_full_trainset()
algo.train(trainset)

# now test on the trainset
testset = data.construct_testset(trainset_raw_ratings)
predictions = algo.test(testset)
print('Accuracy on the trainset:')
accuracy.rmse(predictions)

# now test on the testset
testset = data.construct_testset(test_raw_ratings)
predictions = algo.test(testset)
print('Accuracy on the testset:')
accuracy.rmse(predictions)


PS: If you feel like this seems a bit hacky and weird... then the core-developer of Scikit-learn that wrote this code also agrees with that sentiment.

• Thanks but this is not what I am looking for. I want to split the data into train and test first. Then I want to run cross-validation on the training set. – Ahsan Tarique May 5 '20 at 14:35
• Hi, sorry I just realized what exactly wasn't working, I just rewrote my answer entirely. Hope it works this time. – A Kareem May 5 '20 at 15:21