We have different pre-trained models like BERT, USE, ELMo, Word2Vec, FastText, etc.., we have documents in different sizes (large, medium, small). now, we want to do document similarity. how can we decide which pre-trained model/transformer suits for our requirement (for fine-tuning) and what will be the best approach to do?
$\begingroup$
$\endgroup$
7
-
$\begingroup$ What do you mean by "USE"? $\endgroup$– noeCommented Jun 16, 2023 at 15:12
-
$\begingroup$ Universal Sentence Encoder (USE) (arxiv.org/abs/1803.11175) @noe $\endgroup$– tovijayakCommented Jun 16, 2023 at 16:08
-
$\begingroup$ How long are the documents? $\endgroup$– noeCommented Jun 16, 2023 at 16:29
-
$\begingroup$ have edited question, please refer now. @noe $\endgroup$– tovijayakCommented Jun 16, 2023 at 17:39
-
$\begingroup$ How many clusters do you want or guess they can be clustered into? $\endgroup$– MemristorCommented Jun 18, 2023 at 1:01
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
- I think using transformer's model could be a good choice since it can understand the context well. Also, it can handle any size of data if done correctly.
- About choosing the right model, then there are a lot of pretrained models available on huggingface hub. You can choose any model you want based on your domain and the data that you have.
answered Jun 21, 2023 at 4:48