I started to work with LLMs lately and want to know how people choose their pre-trained models in their fine-tuning tasks? What is the criteria to choose the base model and which factors affect?


1 Answer 1


There are too many! Some examples:

  1. Intended use of the model regarding its compatibility with the licenses of available model (p.ej. is commercial use allowed?).

  2. intended use of the model to know if the model should be instruction-tuned or not.

  3. memory and compute restrictions (e.g. deploying on a mobile device vs. deploying on a huge cloud machine with dozens of GPUs).

  4. languages to support.

  5. potential size of the context window.

  6. Domain-specific issues (p.ej. better to use a model trained on legal texts if you are going to use it for the legal domain).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.