1
$\begingroup$

Could someone help with fine-tuning dolphin-2.2.1?

I have a problem with training: my train\loss - 0 and validation\loss - 0.000... after 800-1000 steps and this is overfitting

Params:
dataset 250k, format "text" ### Human: ### Assistant: 
Prompt: ChatML
Trainer: optim - AdamW(model.parameters(), lr=6e-7, betas=(0.9, 0.95), eps=1e-05,), 
lr_scheduler_type="cosine",
warmup_steps=100,
per_device_train_batch_size=5,
per_device_eval_batch_size=5,
gradient_checkpointing=True,
gradient_accumulation_steps=4,
seed=42,
max_steps=10000,
learning_rate=6e-7,
logging_steps=100,
bf16=True,
logging_dir="./logs",
save_strategy="steps",
save_steps=100,
evaluation_strategy="steps",
eval_steps=100,
do_eval=True

Model with LORA: 
config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "o_proj",
        "gate_proj",
        "up_proj",
        "down_proj",
        "lm_head",
    ],
    bias="none",
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
) 

Not much experience I can't figure out what causes the model weights to be so memorized

More info:
fsdp_plugin = FullyShardedDataParallelPlugin(
    state_dict_config=FullStateDictConfig(offload_to_cpu=True, rank0_only=False),
    optim_state_dict_config=FullOptimStateDictConfig(offload_to_cpu=True, rank0_only=False),
)

accelerator = Accelerator(fsdp_plugin=fsdp_plugin)

base_model_id = "cognitivecomputations/dolphin-2.2.1-mistral-7b"
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

model = MistralForCausalLM.from_pretrained(base_model_id, quantization_config=bnb_config)

tokenizer = LlamaTokenizer.from_pretrained(
    base_model_id,
    padding_side="left",
    add_eos_token=True)

tokenizer.pad_token = tokenizer.eos_token

def tokenize(prompt):
    result = tokenizer(
        prompt,
        truncation=True,
        max_length=2048,
        padding="max_length",
    )
    result["labels"] = result["input_ids"].copy()
    return result

libs: bitsandbytes, github.com/huggingface/transformers.git, github.com/huggingface/peft.git, github.com/huggingface/accelerate.git, datasets scipy ipywidgets latest

name\step\train|loss\eval|loss

dolphin-2.2.1_new-2023-12-21-11-49  250 5.6574  5.372434139251709 
dolphin-2.2.1_new-2023-12-21-17-19  500 4.3343  3.06201434135437 
dolphin-2.2.1_new-2023-12-21-20-06  750 1.8981  0.4487628936767578
dolphin-2.2.1_new-2023-12-22-07-52  1000    0.2334  0.1926171183586121 
dolphin-2.2.1_new-2023-12-22-13-18  1250    0.1687  0.1445329338312149 
dolphin-2.2.1_new-2023-12-22-18-39  1500    0.1213  0.096932053565979 
dolphin-2.2.1_new-2023-12-23-00-00  1750    0.0694  0.039184220135211945
dolphin-2.2.1_new-2023-12-23-06-34  2000    0.0169  0.0011213048128411174 
dolphin-2.2.1_new-2023-12-23-19-08  2250    0.0027  0.00005872833207831718 
dolphin-2.2.1_new-2023-12-24-01-06  2500    0.0009  0.000027619591492111795 
dolphin-2.2.1_new-2023-12-24-10-11  2750    0.0002  0.000020941797629348 
dolphin-2.2.1_new-2023-12-24-16-02  3000    0.0001  0.000017155833120341413 
dolphin-2.2.1_new-2023-12-24-21-51  3250    0.0001  0.00001347742090729298 
dolphin-2.2.1_new-2023-12-25-07-43  3500    0   0.000011896418072865345
PeftModelForCausalLM(
  (base_model): LoraModel(
    (model): MistralForCausalLM(
      (model): MistralModel(
        (embed_tokens): Embedding(32002, 4096)
        (layers): ModuleList(
          (0-31): 32 x MistralDecoderLayer(
            (self_attn): MistralSdpaAttention(
              (q_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (k_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=1024, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=1024, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (v_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=1024, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=1024, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (o_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (rotary_emb): MistralRotaryEmbedding()
            )
            (mlp): MistralMLP(
              (gate_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=14336, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=14336, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (up_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=4096, out_features=14336, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=4096, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=14336, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (down_proj): lora.Linear4bit(
                (base_layer): Linear4bit(in_features=14336, out_features=4096, bias=False)
                (lora_dropout): ModuleDict(
                  (default): Dropout(p=0.05, inplace=False)
                )
                (lora_A): ModuleDict(
                  (default): Linear(in_features=14336, out_features=8, bias=False)
                )
                (lora_B): ModuleDict(
                  (default): Linear(in_features=8, out_features=4096, bias=False)
                )
                (lora_embedding_A): ParameterDict()
                (lora_embedding_B): ParameterDict()
              )
              (act_fn): SiLU()
            )
            (input_layernorm): MistralRMSNorm()
            (post_attention_layernorm): MistralRMSNorm()
          )
        )
        (norm): MistralRMSNorm()
      )
      (lm_head): lora.Linear(
        (base_layer): Linear(in_features=4096, out_features=32002, bias=False)
        (lora_dropout): ModuleDict(
          (default): Dropout(p=0.05, inplace=False)
        )
        (lora_A): ModuleDict(
          (default): Linear(in_features=4096, out_features=8, bias=False)
        )
        (lora_B): ModuleDict(
          (default): Linear(in_features=8, out_features=32002, bias=False)
        )
        (lora_embedding_A): ParameterDict()
        (lora_embedding_B): ParameterDict()
      )
    )
  )
)
$\endgroup$
0

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.