How to use rule-based labelling intelligently?

Question

I have a dataset like below

The outcome column is labelled as positive if the % difference between target final Qty and Booked Qty is less than 50%. Else negative.

But is there any chance to get likelihood of positive or negative? Meaning, my real dataset has more than 10K rows and our biz team cannot follow up with all instances identified as negative and steer them towards becoming positive. If let's say my dataset is imbalanced and 10:90 ratio. We would have to invest too much of resources to convince all this 90% of customers.

Instead, we would like to identify who have the better chance of becoming positive(from negative). So, this can allow us to focus only on selected instances (of that 90%). For ex: We may focus only on customers who show the likelihood of 80% or above (to convert to positive).

As you can see, am not talking about the likelihood for an unseen data point. I am discussing about the actual dataset that I have.

a) Any intelligent way to assign likelihood to this table?

b) Any other simple statistical approach to rank my labels (with or without ML for my existing data)?

c) I understand we can use logistic regression to compute the likelihood. But that can work only for a new unseen customer. I am basically trying to achieve two things a) Label my records in a intelligent way b) compute the likelihood for an unseen data point

Answer 1

a) Any intelligent way to assign likelihood to this table?

Your conversions are essentially an outcome of two functions, your biz team and the procurement team/contact of the client. The common denominator is your biz team.

Consequently, in order to be able to predict likelihood you need recorded metrics of your biz team's both successful and unsuccessful conversions and from that data your model can start identifying the factors affecting the outcomes positively or negatively.

b) Any other simple statistical approach to rank my labels (with or without ML for my existing data)?

Assuming you actually have relevant metrics, you can programmatically cycle through labels and their combinations to see which ones have the highest positive impact on the model's predictive accuracy.