2
$\begingroup$

I am building a project for my bachelor thesis and am wondering how to prepare my raw data. The goal is to program some kind of semantic search for job postings. My data set consists of stored web pages in HTML format, each containing the detail page of a job posting. Via an interface I want to fill in predefined fields like skills, highest qualification, etc. with comma-separated sentences or words. These are then embedded via a Hugging Face Transformer and afterwards the similarity of the input is to be compared with the already embedded job postings and the "best match" is returned.

I have already found that intensive preprocessing such as stop word removal and lemmatization is not necessarily required for transformers. However, the data should be processed to resemble the data on which the pre-trained transformers learned. What would be the best way to prepare such a data set to fine-tune pre-trained Hugging Face Transformers?

Additional info: 55,000 of the saved web pages contain an annotation scheme via which I could simply extract the respective sections "Skills" etc. from the HTML text. If that is not sufficient, I can use prodigy to further annotate the data, e.g. by span labeling texts within the text of the job postings.

Thank you very much in advance!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Resumes are quite different from classic text because there are many proper nouns (names, companies, places, etc.) and other data difficult to classify (phone numbers, marks, age, etc.).

That's why you can use lighter versions like DistilBert to train your data on resumes and get good results.

Therefore, you should first separate every paragraph and label them to classify resumes correctly.

You can also use pre-trained models like this one and fine-tune them with your data.

However, this is not a semantic search yet. After classifying resumes content correctly, you can use a semantic transformer to look for field similarity among the same resumes category.

Note: the computing power might be very high if you have thousands of CVs to compare with, even if you detect the search category and process the comparisons in one category only.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Does it answer your question? If not, please let me know. $\endgroup$
    – Nicolas Martin
    Aug 23, 2022 at 14:08
  • $\begingroup$ Thank you, Nicolas, it helped me. I created the separate paragraphs and their labels, and now I am working on an Encoder-Decoder Sequence to Sequence Model using Attention to reduce the dimensionality of the embeddings for the sentences in each paragraph. It takes a lot of computing otherwise, as you pointed out in your answer. In case you have some experience regarding that as well, I have already posted a question to it here. $\endgroup$
    – nesquick
    Sep 1, 2022 at 12:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.