I have a binary classification model $A$ and want to calculate its precision and recall for positive and negative classes. The "ground truth" or "true" labels for this model are obtained from another binary classification model $B$, whose precision and recall for both classes I know.

I'm almost sure that the metrics for $A$ will be probabilistic in nature, but I have no idea on how to calculate them, given the performance metrics of $B$. As far as I understand, if I denote the true label by $y$, the label predicted by $B$ as $y_B$ and the label predicted by $A$ as $y_A$, then I have to calculate $P(y_A=i\ |\ y=j)$ where $i,j\in\{0,1\}$. And of course, I know the values of $P(y_A=i\ |\ y_B=j)$ for all $i,j$. So

$$P(y_A=i\ |\ y=j)=\frac{P(y_A=i\ \cap\ y=j)}{P(y=j)}=\frac{P(y_A=i\ \cap\ y=j)}{P(y=j\ \cap\ y_B=1)+P(y=j\ \cap\ y_B=0)}$$

The denominator terms on the RHS can be estimated from the precision-recall metrics of model $B$, but I have no idea how to deal with numerator. Maybe an entirely different approach is needed. Would appreciate any help in this regard!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.