I have 4 class binary classification models. That models identify which class a particular students is suitable for.
For example, we have
user 1 and 4 classes recommendation model.
Models were identify how this user would like to take its class.
user 1's personal profile data (features), model A, B, C, D predicts each class' fitness. Binary classification threshold were all 50%.
model A: 77%, True
model B: 65%, True
model C: 33%, False
model D: 88%, True
Based on this result, system recommends class
A, B, and C to
However, models' performance were all different. Each model may have different F1-score, for example,
model A: 77%,
model B: 64%,
model C: 81%, and
model D: 55%.
How can we measure each recommendation score rationally, based on models' F1 score?
I also had thought that some recommender system might works, however recommendation algorithms were limit to utilize user's profile.