0
$\begingroup$

I come here cause I have some troubles (or is it normal ?) with the rating predicted by SVD() and KNNWithMeans(), I'm using the Sckit-Surprise library . Here is context :

  • I have 637 069 rating
  • I have 2 101 users
  • And I have 5 870 items
  • So in average, each user rate 300+ items

(This is a subset, I can take the whole dataset which is 4-5 time bigger, it will increase accuracy ?)

All the rating are between 0 and 1 (that I have computed from history data) : here is an exemple of rating dataframe :

enter image description here

I have few questions : first about the RMSE of SVD() : I do this :

trainset = data.build_full_trainset()
algo = SVD()
algo.fit(trainset)
# Than predict ratings for all pairs (u, i) that are NOT in the training set.
testset = trainset.build_anti_testset()
predictions = algo.test(testset)

accuracy.rmse(predictions, verbose=True)`

And I get :

RMSE: 0.2327
0.23271217680660156

And when I perform a cross validation :


algo = SVD()
# We can now use this dataset as we please, e.g. calling cross_validate
cross_validate(algo, data, verbose=True)

And now, I'm getting :

enter image description here

Do this difference is normal ? Do these score are good in regard of the score between 0-1 ?

My other question is about the difference between prediction with SVD() and KNNWithMeans() :

I perform both :

algo = SVD()
output = algo.fit(data.build_full_trainset())

and


algo = KNNWithMeans()
output = algo.fit(data.build_full_trainset())

And after I retrieve estimation for items with this function :


def get_estimation_user(user):
   listeProduit = notation['Item'].unique()
   estimations = []
   for produit in listeProduit:
    estimations.append(output.predict(str(user),produit).est)

   df_estimation = pd.DataFrame(
     {'Item': listeProduit,
      'estimation': estimations
     })

  return df_estimation

I use this function with a particular user, and the same user in both SVD and KNN And, with SVD(), I get that :

enter image description here

And KNN, I get that :

enter image description here

This difference is normal ? I think it is because of the way that KNN works but I want to have the opinion of more specialized peoples.

The good point is that, for the same item, the result are almost the same :

enter image description here

(This is the top rated item on SVD, with a score of 0.672791

But for the top rated item by KNN :

enter image description here (This item is rated with a score of 1.0 by KNN)

Why these differences ? It is a good idea to transform these rating to a "finite scale" like 0.05,0.10,0.15,0.20,... ?

When I use one algorithm, can I get some "justification" for a prediction ? like this rating was predicted because the user is similar to this user and this user have a good rate ? Versions

I run on Google Colab

Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic

Python 3.6.9 (default, Apr 18 2020, 01:56:04)

[GCC 8.4.0]

surprise 1.1.0

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.