# What techniques should I use to compare the similarity between a bunch of texts?

If I have a list of job postings stored as raw texts and I want to compare the similarity of all the job postings to a given resume, what techniques or algorithms should I use?

I'm thinking of starting with a model that converts the job postings into a vector space using TF-IDF and then just calculate the matrix cosine similarity. Can I improve on this?

As of now, I can think of two ways to formulate this problem:

# 1. Search problem

Parse your job listings and index them in some sort of search engine like Solr or ElasticSearch. You can build capabilities like Semantic search using Word2Vec models, etc.

Now write a query engine which takes a resume and queries this Search engine. It will be blazing fast since job listing will be all indexed.

# 2. Similarity problem

I would have created hybrid similarity function. For example:

a) How many top key words matched between resume and job listing

b) Similarity of resume and job listing using Doc2Vec (Pre compute vector for job listings)

c) Using algorithms like Locality Sensitive Hashing to reduce the lookup space

This approach will be slow but might yield a good result.

Start with named entity recognition, it gives an idea which part of posting is requirements and what part is description and then you could try just with bag of words. As the topic structure and vocabulary is standard and no weights are needed for word importance, I dont find a reason for TF-IDF to be used here.