I am working for a recruitment company on developing machine learning algorithms to automatically classify job applicants as either to be interviewed or not be interveiwed. The data is highly imbalanced (i.e most applicants will not be interviewed.) Currently, to do this we use a machine learning model per job. We feed it examples of applicants with their status. However, this has generally low performance due to the small size of the training dataset.

A potential improvement on this approach could be creating a single model which takes in as input information on the applicant and information on requirements of the job and then predicting whether or not to interview. Is this approach likely to work and a technique that is used/has been researched in any context?

Note: the actual problem I am working on is to do with classifying publications for inclusion in medical systematic reviews. The 'different jobs' represent different review topics, the 'job requirements' are the criteria for inclusion and each 'applicant' is actually a medical publication. I used this example as it is comparable and requires less domain expertise to understand.

  • 1
    $\begingroup$ It's wrong to do this... $\endgroup$
    – Aditya
    Jun 7 '18 at 12:49
  • 3
    $\begingroup$ With multiple models, you mean models corresponding to different job poster? $\endgroup$ Jun 7 '18 at 13:05
  • 1
    $\begingroup$ @Aditya Why is this wrong? $\endgroup$
    – jdoe
    Jun 7 '18 at 13:22
  • 1
    $\begingroup$ I meant it isn't right to use a machine to judge CV's, being a student that's my opinion as we don't me tion Everything on the CV, there a reason lot of things which we want the interviewer to ask us.. $\endgroup$
    – Aditya
    Jun 7 '18 at 14:53
  • $\begingroup$ @Aditya The actual problem doesn't have anything to do with jobs - its in the medical space with evaluating research publications for inclusion in systematic reviews but I used this as it is comparable and requires less domain expertise to understand... but I do agree in the case of jobs. $\endgroup$
    – jdoe
    Jun 7 '18 at 16:59

This questions is best posed in its original domain, ie medicine, because it will require significant domain knowledge about the problem structure and the nature of the data to reason about how well a single model might do. I doubt the analogy to job applications translates perfectly here.

Right now you have one model per review topic which is able to generalize to different publications with different criteria for inclusion. You are asking whether a single model will also be able to generalize to different review topics.

Of course you could build the model and evaluate its accuracy. If that will require significant effort, then it is fair try and reason about how well you might do first, but that will only get you so far.

Again you will have to apply your own domain knowledge here, but I can at least try and give you a framework for thinking about this problem. In order to answer the main question of how well one model will generalize, you need to investigate:

  • Do different review topics interact with publications / inclusion criteria in similar ways?
  • Do any of my existing features capture the interaction between review topics and publications / inclusion criteria?
  • Are there any additional features I can design to capture these interactions in a general way?

To give a very simple example, if having the topic keyword x number of times in the paper was sufficient for inclusion for paper X, topic Y, publication Z with review criteria A,B,C,..., and you think this data helps you predict inclusion for a completely different paper, topic, etc. example, then you might reason that one model will generalize here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.