I am working with large datasets of papers and authors. I am trying to find top-k authors that are likely to cite a new paper on the unseen dataset (https://www.aminer.org/aminernetwork). My setup is Pyspark for parallel processing. Here is the overview for the datasets:
Paper dataset, the middle is the fact table and then author table.
My idea was to create features from both datasets and find similarities between vectors of features. I am not sure how to define that for any of the models in machine learning.
Since I need both paper and author ids as well as features vector, I was thinking in the direction of the recommender system. In this case ALS is supported in Pyspark, but does not take vector of features but ratings as numeric values.
Please any tips and hints are welcome!