0
$\begingroup$

For my task, I need a model that can distinguish between job titles that contain the same words. BERT model "msmarco-MiniLM-L-12-v3" shows high cosine similarity for positions: "Data customer" and "Data provider". The meaning of these two positions are very different and I need my model to show a low cosine similarity for these two positions.

However, in this case cosine similarity must be high: "Data customer" "Data consumer".

Which model should I use? Should I train classifier instead of nlu model? Why ChatGPT understands the difference between those texts, but BERT based models show high cosine similarity?

$\endgroup$

1 Answer 1

0
$\begingroup$

Using the hosted inference here: https://huggingface.co/sentence-transformers/msmarco-MiniLM-L-12-v3 I'm getting a sentence similarity of 0.58 between data customer and data provider, and 0.86 between data customer and data consumer, which seems pretty reasonable to me. enter image description here

If you're willing to train the model from scratch you could consider negative sampling https://www.baeldung.com/cs/nlps-word2vec-negative-sampling.

Using a classifier algorithm (SVMs or k-Nearest Neighbors) for a finite number of job titles would probably be a reasonable starting point, and you could use your embeddings as a starting point. hth.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.