Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
If I have a list of job postings stored as raw texts and I want to compare the similarity of all the job postings to a given resume, what techniques or algorithms should I use?
I'm thinking of starting with a model that converts the job postings into a vector space using TF-IDF and then just calculate the matrix cosine similarity. Can I improve on this?
As of now, I can think of two ways to formulate this problem:
Parse your job listings and index them in some sort of search engine like Solr or ElasticSearch. You can build capabilities like Semantic search using Word2Vec models, etc.
Now write a query engine which takes a resume and queries this Search engine. It will be blazing fast since job listing will be all indexed.
I would have created hybrid similarity function. For example:
a) How many top key words matched between resume and job listing
b) Similarity of resume and job listing using Doc2Vec (Pre compute vector for job listings)
c) Using algorithms like Locality Sensitive Hashing to reduce the lookup space
This approach will be slow but might yield a good result.
Start with named entity recognition, it gives an idea which part of posting is requirements and what part is description and then you could try just with bag of words. As the topic structure and vocabulary is standard and no weights are needed for word importance, I dont find a reason for TF-IDF to be used here.
