We have a very imbalanced dataset (2% of class 1). To the best of our knowledge, there is no baseline in the literature to the problem we want to solve - so we thought of comparing our performance to a random classifier. We evaluate our model as a combination of precision and recall - we vary the threshold at which data points are classified as 1 and compute the rolling threshold and recall. We could use F1-score as well.
What would be an acceptable way to define a random predictor that we can compare to our model such that the comparison is as fair as possible?