Best approach to find top-k most likely items

Question

I am working with large datasets of papers and authors. I am trying to find top-k authors that are likely to cite a new paper on the unseen dataset (https://www.aminer.org/aminernetwork). My setup is Pyspark for parallel processing. Here is the overview for the datasets:

Paper dataset, the middle is the fact table and then author table.

My idea was to create features from both datasets and find similarities between vectors of features. I am not sure how to define that for any of the models in machine learning.

Since I need both paper and author ids as well as features vector, I was thinking in the direction of the recommender system. In this case ALS is supported in Pyspark, but does not take vector of features but ratings as numeric values.

Please any tips and hints are welcome!

Answer 1

Basic idea is ALS plays with a matrix of each row represents (e.g.) an author and each column represents (e.g.) a paper. The value of a specific row and specific column represents the "interaction" between the authoer and the paper. It can be rating as you mentioned which has a scale from 0 to 5, or it can be binary (e.g. like / not like) which means 0 or 1. I think in your situation, the interaction would be more like the binary case, which is cited or not cited.