Approaches for matching leads to salesmen

Question

I'm starting to tackle a new problem where we are trying to optimally match new leads (perspective customers) for our product to our sales representatives in the hopes of improving bottom-line metrics like conversion rate, average sale price, etc. We have a bunch of data from the leads when they fill out their info on web forms and from 3rd party data providers we use to enrich the core web form data (we try and pull their soft credit score, income, etc. based on the info they provide, this is all automated).

On the salesman side, we don't have nearly as much data on them (mainly just who they are and their sales performance history). I suppose we could actually run them through our data enrichment service to pull additional info on them though.

My question is simply: from an ML perspective what, would be the best way to structure this problem? I was thinking of just building models for each salesman and assigning the lead to the salesman with the highest predicted score (e.g. for conversion) but this seems a bit crude. I was also considering recommender systems given the matching nature of the problem but my background is more in traditional ML so not sure what subtype would be best to start with (content-based, collaborative, etc.).

Any input is greatly appreciated.

Answer 1

I'm going to make a bunch of assumptions about the shape of your data and model choice, just to make the setup simple and concrete. Hopefully the broader ideas will generalize from there.

Suppose you wrangle your data into a matrix with a response vector of zeros and ones representing whether a sale was made. This is a nice simple supervised classification problem and logistic regression is probably the first thing you try in this case.

If you ignore which sales rep each customer was assigned to, you will get a model that tells you the probability of a sale based on customer characteristics (income, etc). But it won't tell you anything about which is the best sales rep to assign.

If you fit a separate model for each sales rep then you could compare the outputs of each model. But I share your concern about this approach. Each model could pick up some idiosyncrasies. Also, if a sales rep got lucky with getting good leads in the past then they are likely to fit to a model with a high general probability of sale for all leads simply because the constant term is higher in their models than others.

There might be another constraint in your ideal system here -- presumably you want to avoid building a model that just assigns all leads to the best sales rep.

Another approach would be to fit a single logistic regression but include instrumental (or dummy) variables for the sales reps. A column for each rep with a one if they worked with that customer and a zero otherwise. Your coefficients vector will then include a coefficient for each sales rep. This feels like a step in the right direction but will necessarily result in a model where all leads are assigned to a single sales rep -- to the one with the highest coefficient. One nice aspect to this approach is that it requires no additional information about the sales reps. Only information about which customers they previously worked with is needed.

A next step from there might be to add cross-terms. That is, features that are the product of a customer characteristic and a sales rep indicator. This isn't guaranteed produce a model where leads are assigned evenly between sales reps but might produce recommendations of the form "assign low income customers to rep A and high income customers to rep B". (Whether or not such a recommendation is politically acceptable in your firm is a different question entirely.)

I'm not sure that model is going to get you to something that you would be willing to use in production but it might be a nice first step to get a sense of the data and which variables tend to be predictive.

One last thought: your dataset might include some information that precludes some sales reps working with some customers. The rep and the customer have to be in the same geographic region, perhaps. You're definitely going to want to work that in somehow. If customers and sales reps are split into disjoint regions then you just fit a model per region. If it is more complicated than that then your model will be, inevitably, more complicated.