We have different pre-trained models like BERT, USE, ELMo, Word2Vec, FastText, etc.., we have documents in different sizes (large, medium, small). now, we want to do document similarity. how can we decide which pre-trained model/transformer suits for our requirement (for fine-tuning) and what will be the best approach to do?
- I think using transformer's model could be a good choice since it can understand the context well. Also, it can handle any size of data if done correctly.
- About choosing the right model, then there are a lot of pretrained models available on huggingface hub. You can choose any model you want based on your domain and the data that you have.