Skip to main content
Data Science

Return to Question

Grammar, Case correctiona dn spelling mistakes
Source Link
edit approved Jun 2 at 15:18
Aman Mathur
edit approved Jun 2 at 15:18
Aman Mathur
  • 225
  • 1
  • 10

I have a dataset that iI have collectcollected for specific topic the.

The dataset is in thesethe following format  :

  1. rawRaw text (similar to shake spareshakespeare dataset) where it has no label or input, just text
  2. Question and answer dataset similar to alpaca instructs

myMy way is to fine tune a LLM on raw text first then on Q&A dataset

isDoes this looks like a good practice approach? orOr can iI just fine tune it on all the datasets together  ?

I have a dataset that i have collect for specific topic the dataset is in these format  :

  1. raw text (similar to shake spare dataset) where it has no label or input, just text
  2. Question and answer dataset similar to alpaca instructs

my way is to fine tune a LLM on raw text first then on Q&A dataset

is this looks like a good practice ? or can i just fine tune it on all the datasets together  ?

I have a dataset that I have collected for specific topic.

The dataset is in the following format:

  1. Raw text (similar to shakespeare dataset) where it has no label or input, just text
  2. Question and answer dataset similar to alpaca instructs

My way is to fine tune a LLM on raw text first then on Q&A dataset

Does this looks like good approach? Or can I just fine tune it on all the datasets together?

Source Link
Mustafa Alahmid
Mustafa Alahmid
  • 305
  • 1
  • 9

Best practice for fine tuning LLM

I have a dataset that i have collect for specific topic the dataset is in these format :

  1. raw text (similar to shake spare dataset) where it has no label or input, just text
  2. Question and answer dataset similar to alpaca instructs

my way is to fine tune a LLM on raw text first then on Q&A dataset

is this looks like a good practice ? or can i just fine tune it on all the datasets together ?