# How do I estimate the benefit of one business method over another when all my data is confounded?

I'm trying to estimate the benefit of using one method over another, but the existing data is confounded and I'm struggling to figure it out.

So my company submits debit orders to customers to collect a monthly payment. Sometimes customers reverse the charge or don't have enough money, so the payment fails and we don't collect any money from them that month.

Orders are submitted through method A, which is cheap but can be reversed; and through method B, which is more expensive but can't be reversed, although it can still fail. Overall method B is more likely to succeed in collecting a payment, and I'm trying to estimate how much better it is. How much more likely are we to collect a payment if method B is used over method A?

Currently a simple rule based system is used to decide whether to use method A / B for the next payment, which is based off of how many payments the customer previously failed. So the worse customers are more likely to have had their payments made on method B.

So when I look at my payments data, the average payment rate for method A is say 90% and the average payment rate for method B is say 70%. That makes sense, since the customers who were more likely to not pay are assigned to method B. But we know that method B is better at collecting payments!

So my issue is, how do I quantify how much better method B is from this data?