I am working on a problem for which no datasets exist. I have obtained several examples from this domain, and so far have been using them in Large Language Model (LLM) prompts(few shot learning) but I noticed results are not good and perhaps finetuning is the way to go. The expected output of LLM is to generate text and logical formulas.

I am looking for tutorials that include best practises for the following and more:

  1. How to decide which fields (columns) to include?
  2. What types of data to include as input?

Any pointers is extremely appreciated.

  • $\begingroup$ What do you mean by "fields" and "types of data"? LLMs are trained and fine-tuned on raw text. I think more information is needed to understand your use case. $\endgroup$
    – noe
    Mar 20 at 8:03
  • $\begingroup$ @noe please see an example (datacamp.com/tutorial/fine-tuning-large-language-models). There are 4 fields in the dataset. Label is 0/1 and the rest are text. I am looking for best practises to teach me why for example, these specific 4 columns were chosen? $\endgroup$
    – Karl 17302
    Mar 20 at 9:16
  • $\begingroup$ The four columns are just what happen to come with the dataset. They use text and labels because they are specifically training a text classification model. It's not a model predicting text, it's a model that maps input text to specific classes. $\endgroup$
    – Karl
    Mar 21 at 1:30


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.