0
$\begingroup$

I have a binary classification model $A$ and want to calculate its precision and recall for positive and negative classes. The "ground truth" or "true" labels for this model are obtained from another binary classification model $B$, whose precision and recall for both classes I know.

I'm almost sure that the metrics for $A$ will be probabilistic in nature, but I have no idea on how to calculate them, given the performance metrics of $B$. As far as I understand, if I denote the true label by $y$, the label predicted by $B$ as $y_B$ and the label predicted by $A$ as $y_A$, then I have to calculate $P(y_A=i\ |\ y=j)$ where $i,j\in\{0,1\}$. And of course, I know the values of $P(y_A=i\ |\ y_B=j)$ for all $i,j$. So

$$P(y_A=i\ |\ y=j)=\frac{P(y_A=i\ \cap\ y=j)}{P(y=j)}=\frac{P(y_A=i\ \cap\ y=j)}{P(y=j\ \cap\ y_B=1)+P(y=j\ \cap\ y_B=0)}$$

The denominator terms on the RHS can be estimated from the precision-recall metrics of model $B$, but I have no idea how to deal with numerator. Maybe an entirely different approach is needed. Would appreciate any help in this regard!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.