I come here cause I have some troubles (or is it normal ?) with the rating predicted by SVD() and KNNWithMeans(), I'm using the Sckit-Surprise library . Here is context :
- I have 637 069 rating
- I have 2 101 users
- And I have 5 870 items
- So in average, each user rate 300+ items
(This is a subset, I can take the whole dataset which is 4-5 time bigger, it will increase accuracy ?)
All the rating are between 0 and 1 (that I have computed from history data) : here is an exemple of rating dataframe :
I have few questions : first about the RMSE of SVD() : I do this :
trainset = data.build_full_trainset()
algo = SVD()
algo.fit(trainset)
# Than predict ratings for all pairs (u, i) that are NOT in the training set.
testset = trainset.build_anti_testset()
predictions = algo.test(testset)
accuracy.rmse(predictions, verbose=True)`
And I get :
RMSE: 0.2327
0.23271217680660156
And when I perform a cross validation :
algo = SVD()
# We can now use this dataset as we please, e.g. calling cross_validate
cross_validate(algo, data, verbose=True)
And now, I'm getting :
Do this difference is normal ? Do these score are good in regard of the score between 0-1 ?
My other question is about the difference between prediction with SVD() and KNNWithMeans() :
I perform both :
algo = SVD()
output = algo.fit(data.build_full_trainset())
and
algo = KNNWithMeans()
output = algo.fit(data.build_full_trainset())
And after I retrieve estimation for items with this function :
def get_estimation_user(user):
listeProduit = notation['Item'].unique()
estimations = []
for produit in listeProduit:
estimations.append(output.predict(str(user),produit).est)
df_estimation = pd.DataFrame(
{'Item': listeProduit,
'estimation': estimations
})
return df_estimation
I use this function with a particular user, and the same user in both SVD and KNN And, with SVD(), I get that :
And KNN, I get that :
This difference is normal ? I think it is because of the way that KNN works but I want to have the opinion of more specialized peoples.
The good point is that, for the same item, the result are almost the same :
(This is the top rated item on SVD, with a score of 0.672791
But for the top rated item by KNN :
(This item is rated with a score of 1.0 by KNN)
Why these differences ? It is a good idea to transform these rating to a "finite scale" like 0.05,0.10,0.15,0.20,... ?
When I use one algorithm, can I get some "justification" for a prediction ? like this rating was predicted because the user is similar to this user and this user have a good rate ? Versions
I run on Google Colab
Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0]
surprise 1.1.0