Consider a talent pool in which each member has some set of skills. Some of these talent are submitted to orders as potential candidates of which one is selected. It is reasonable to assume that the submitted talent have some dominant thing in common in their skill sets (let's call it a segment) that qualifies them for the order. Example segments are "front end web-designer" or "brochure / sprint designer".

Given the total set of skills over all of the talent submitted to an order (like 2-5 with say 10 skills each, so 20 - 50 skills total), I am looking for the dominant segment. Then, I am looking for the dominant segment for each individual talent.

My plan is to use latent Dirichlet allocation (LDA) such that the skills of all the talent submitted for an order are a "document" that contains some segments or "topics" with some probability. Likely, there will be one or two dominant topics depending on the total topic number. I will then use this model to predict the dominant segment for each talent where the individual talent skill set is a "document" with some segments or "topics" within.

I am curious if anyone has feedback about my use of LDA or other ideas about how I might go about discovering these segments?


1 Answer 1


You may want to consider preprocessing - convert different wording for the same type of talent to same wording. For example, a talent in machine learning is called data scientist at Coursera job site, data engineer at Udacity job site, or data analyst. This preprocessing is similar to stemming in concept.

  • $\begingroup$ Not a bad idea, although in that particular case I think I'd want separation. Things like "software developer" and "software engineer" might be close enough to merge (in our case, maybe not others). $\endgroup$
    – Chris
    May 25, 2015 at 18:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.