Machine learning worker performance features for optimum allocation of tasks to workers

Question

Suppose we have an English to French translation task in a company, and there are 100s of workers who are proficient in doing this task, but each worker has its own unique attributes which enables them to do certain translations in a better way than the others (e.g., a translator who is a doctor would do better translations of medical documents). Consider 1000s of documents for translation received per day, and each document takes one day to translate. So, there is a two-fold problem:

1. Allocating the tasks to available translators using a queuing model or any other efficient allocation mechanism.
2. Learning the worker model based on the past performance of each (e.g., correctness scores, static parameters like skill set, qualifications, experience, etc.) using a machine learning system.

Is there a setup/system/model which solves both of the above problems in one cohesive system? From my reading, this problem calls for the application of the assignment problem, user modelling, queuing, and machine learning optimization. I am searching for existing models or frameworks which integrate all of this, can anyone give me some direction for this task of optimum allocation?

Answer 1

Your problem is going a lot of directions.

If you have known worker attributes, like doctor, engineer, or accountant, you could have mirror document classification, and use machine learning to classify.

If you are going to measure performance, you will need to get a sample from each user. For example, take the top $\frac{1}{4}$ performance and have a target vector for the user. Map a document to a user's strength. If that user is over-queued, pick the next highest $\frac{1}{4}$ and so on. You could use cosine similarity to give you a score. This can get tricky, as you may want to normalize on productivity. You may get a document that a high productivity user is better at, but a low productivity user is OK at. By way of analogy, an american football player may be the best receiver and half back, but you might have a good backup receiver and no good backup halfback, so you let your backup play receiver.

Static parameters would be different, and I am not sure you should use them if you have a performance measure. My guess would be to queue before you have performance for a worker. But even at 4 documents, you can select a top $\frac{1}{4}$.

You could also create clusters and measure a user's performance versus the cluster. This is the easiest approach to queuing, as you just take the next document out of the cluster that is a user's best cluster. You only need a few cross-cluster workers to level queues. The downside is, if you have 20 clusters, it would take a while to get user performance versus cluster, so you would need to use static attributes until you get a valid sampling.