1
$\begingroup$

I was trying to use this project :

https://github.com/UKPLab/sentence-transformers

for embedding non english sentences, the language is not a human speaking language, its machine language (x86)

but the problem is i cannot find a simple example where it shows how can i embed sentences using a custom dataset without any labels or similarity values of the sentences.

basically i have an array of sentences lists without any labels for sentences or similarity values for them, and i want to embed them into vectors in a way that it preserves the semantic of the sentence the best way possible, so far i have used word2vec and doc2vec using gensim library so i wanted to try this method to see if its any better?

Improve this question
$\endgroup$
1
$\begingroup$

The link you provided of Siamese Bert is an instance of a Bert or Roberta finetuned on STS or NLI data. Which can have the format sentence 1 is similar 3 out of 5 to sentence 2 (STS). Hence, is supervised, it does not fit your purpose.

Nonetheless, do not despair, there are some that do not require training, although may not perform as good as the supervised one. The below use word embeddings which you can train on your data corpora to generate sentence embeddings:

Or by feeding just sentences line by line:

P.S. I have not tried all of the solutions, to my knowledge I suggest these, cause either they are quite known or are quite recent.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks for answer, So how do the embedding methods you mentioned compare to Doc2vec or using Word2vec and then averaging the vectors? has there been any sort of benchmark to show which one performs better most of the times? $\endgroup$ – OneAndOnly Aug 25 '20 at 16:20
  • $\begingroup$ Averaging the vectors is not desired, cannot retain the same semantic values. Read the first link and maybe try it, it shows a simple method where it compares the similar words of each sentence to create distances. There are some benchmarks like SentEval, however, methods really depend on the domain, especially yours is a computing language. I would suggest to try the simpler solution then move forward, if you find any good benchmark on non Natural Language you can also let me know, and please if my answer helped you do not forget to act accordingly. $\endgroup$ – Grzegorz Aug 26 '20 at 8:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.