If you have a technical background, I would certainly recommend the chapter 9 from Mining Massive Datasets by Jure Leskovec. Video lecture of the topic is also available there. Look for chapter 9. This blog could also give you some overall insight.
Please keep in mind that, ML-wise, recommendation is a more creativity-related topic than other classic ...
EDIT: It seems I misunderstood the task at first, so here's my correction. Hope it works this time
It seems like what you're trying to do is similar to what is in the documentation under examples/split_data_for_unbiased_estimation.py (or this github issue which seems to be exactly what you want)
The code manually splits the dataset into two without using ...
If the features are identical, good start would be to use n-neighbors approach.
It would be something like that
from sklearn.neighbors import NearestNeighbors
all_songs_features = [[0, 0, 2], [1, 0, 0], [0, 0, 1], [100, 100, 100], [0, 0, 1.5]]
neigh = NearestNeighbors()
my_song_features = [[0, 0, 1.3], [1.1, 0, 0]]
There is always a bias in offline testing in recommender systems. Real evaluation happens in A/B testing. But, that should not discourage one from doing offline testing.
There is ongoing research on this topic using multi-armed bandits. I recommend reading offline testing procedures using reco-gym.
Following workshop talks about removing these biases and ...