0
$\begingroup$

We have different pre-trained models like BERT, USE, ELMo, Word2Vec, FastText, etc.., we have documents in different sizes (large, medium, small). now, we want to do document similarity. how can we decide which pre-trained model/transformer suits for our requirement (for fine-tuning) and what will be the best approach to do?

$\endgroup$
7
  • $\begingroup$ What do you mean by "USE"? $\endgroup$
    – noe
    Jun 16, 2023 at 15:12
  • $\begingroup$ Universal Sentence Encoder (USE) (arxiv.org/abs/1803.11175) @noe $\endgroup$
    – tovijayak
    Jun 16, 2023 at 16:08
  • $\begingroup$ How long are the documents? $\endgroup$
    – noe
    Jun 16, 2023 at 16:29
  • $\begingroup$ have edited question, please refer now. @noe $\endgroup$
    – tovijayak
    Jun 16, 2023 at 17:39
  • $\begingroup$ How many clusters do you want or guess they can be clustered into? $\endgroup$
    – Memristor
    Jun 18, 2023 at 1:01

1 Answer 1

0
$\begingroup$
  • I think using transformer's model could be a good choice since it can understand the context well. Also, it can handle any size of data if done correctly.
  • About choosing the right model, then there are a lot of pretrained models available on huggingface hub. You can choose any model you want based on your domain and the data that you have.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.