0
$\begingroup$

I have fine tune RLHF with Vertex AI Pipeline. But deployed model not showing in model registry. Why?

Piplene templete of fine-tuning

code i have used:

!pip install google-cloud-pipeline-components
!pip install kfp

from google_cloud_pipeline_components.preview.llm import rlhf_pipeline

from kfp import compiler

RLHF_PIPELINE_PKG_PATH = "rlhf_pipeline.yaml"

compiler.Compiler().compile(
    pipeline_func=rlhf_pipeline,
    package_path=RLHF_PIPELINE_PKG_PATH)

parameter_values = {
    "preference_dataset":"bucket link of jsonl file of pref dataset",
    "prompt_dataset":"bucket link of jsonl file of prompt dataset",
    "eval_dataset":"bucket link of jsonl file of eval dataset",
    "large_model_reference":"llama-2-7b",
    "reward_model_train_steps":10,
    "reinforcement_learning_train_steps":5,
    "reward_model_learning_rate_multiplier":1.0,
    "reinforcement_learning_rate_multiplier":1.0,
    "kl_coeff":0.1,
    "instruction":"Give Cultural Proficiency Continuum class: \n\n Explaination:"
}

from google.auth.transport.requests import Request
from google.oauth2.service_account import Credentials

key_path = 'ai-model-408715-690c22f7c53c.json'

credentials = Credentials.from_service_account_file(
    key_path,
    scopes=['https://www.googleapis.com/auth/cloud-platform'])

if credentials.expired:
    credentials.refresh(Request())
PROJECT_ID = 'ai-model-id'
REGION = 'us-central1'

import google.cloud.aiplatform as aiplatform

# Initialize vertex
aiplatform.init(project = PROJECT_ID, location = REGION, credentials = credentials)

job = aiplatform.PipelineJob(
    display_name = 'vsort_rlhf_pipeline',
    template_path = RLHF_PIPELINE_PKG_PATH,
    pipeline_root = 'gs://ai_model_training_bucket/vsort_data/pipeline_root',
    parameter_values=parameter_values
    )

job.run()
$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.