How to train-test split and cross validate in Surprise?

Question

I wrote the following code below which works:

from surprise.model_selection import cross_validate

cross_validate(algo,dataset,measures=['RMSE', 'MAE'],cv=5, verbose=False, n_jobs=-1)

However when I do this: (notice the trainset is passed here in cross_validate instead of whole dataset)

from surprise.model_selection import train_test_split
trainset, testset = train_test_split(dataset, test_size=test_size)
cross_validate(algo, trainset, measures=['RMSE', 'MAE'],cv=5, verbose=False, n_jobs=-1)

It gives the following error:

AttributeError: 'Trainset' object has no attribute 'raw_ratings'

I looked it up and Surprise documentation says that Trainset objects are not the same as dataset objects, which makes sense.

However, the documentation does not say how to convert the trainset to dataset.

My question is: 1. Is it possible to convert Surprise Trainset to surprise Dataset? 2. If not, what is the correct way to train-test split the whole dataset and cross-validate?

Answer 1

EDIT: It seems I misunderstood the task at first, so here's my correction. Hope it works this time

It seems like what you're trying to do is similar to what is in the documentation under examples/split_data_for_unbiased_estimation.py (or this github issue which seems to be exactly what you want)

The code manually splits the dataset into two without using any sort of function call. Then sets the internals of the data variable to be only the train split.

import random                                                              

from surprise import SVD                                                   
from surprise import Dataset                                               
from surprise import accuracy                                              
from surprise import GridSearch                                            


# Load your full dataset.                                                  
data = Dataset.load_builtin('ml-100k')                                     
raw_ratings = data.raw_ratings                                             

# shuffle ratings if you want                                              
random.shuffle(raw_ratings)                                                

# 90% trainset, 10% testset                                                
threshold = int(.9 * len(raw_ratings))                                     
trainset_raw_ratings = raw_ratings[:threshold]                             
test_raw_ratings = raw_ratings[threshold:]                                 

data.raw_ratings = trainset_raw_ratings  # data is now your trainset       
data.split(n_folds=3)                                                      

# Select your best algo with grid search. Verbosity is buggy, I'll fix it. 
print('GRID SEARCH...')                                                    
param_grid = {'n_epochs': [5, 10], 'lr_all': [0.002, 0.005]}               
grid_search = GridSearch(SVD, param_grid, measures=['RMSE'], verbose=0)    
grid_search.evaluate(data)                                                 

algo = grid_search.best_estimator['RMSE']                                  

# retrain on the whole train set                                           
trainset = data.build_full_trainset()                                      
algo.train(trainset)                                                       

# now test on the trainset                                                 
testset = data.construct_testset(trainset_raw_ratings)                     
predictions = algo.test(testset)                                           
print('Accuracy on the trainset:')                                         
accuracy.rmse(predictions)                                                 

# now test on the testset                                                  
testset = data.construct_testset(test_raw_ratings)                         
predictions = algo.test(testset)                                           
print('Accuracy on the testset:')                                          
accuracy.rmse(predictions)  

PS: If you feel like this seems a bit hacky and weird... then the core-developer of Scikit-learn that wrote this code also agrees with that sentiment.