0
$\begingroup$

I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I figured its because of the catastrophic forgetting of the network. So,I would like to save my model when the mean episodic reward becomes maximum.

Example of a training process. enter image description here

So here I would like to save the model when the training rewards are higer. My code


import pandas as pd
from stable_baselines.common.policies import MlpPolicy
from stable_baselines.common.policies import FeedForwardPolicy, register_policy
from stable_baselines.common.vec_env import DummyVecEnv
from stable_baselines import PPO2
from Env import Env
import matplotlib.pyplot as plt
from stable_baselines.logger import configure


configure()

# multiprocess environment
n_cpu = 8
env = DummyVecEnv([lambda: AHUenv() for i in range(n_cpu)])

# Custom MLP policy of three layers of size 128 each
class CustomPolicy(FeedForwardPolicy):
    def __init__(self, *args, **kwargs):
        super(AhuCustomPolicy, self).__init__(*args, **kwargs,
                                           net_arch=[dict(pi=[128, 128],
                                                          vf=[128, 128])],
                                           feature_extraction="mlp")

# Register the policy, it will check that the name is not already taken
register_policy('CustomPolicy', CustomPolicy)



model = PPO2(CustomPolicy,env,gamma=0.9, n_steps=88, ent_coef=0.01,
             learning_rate=2.5e-4, vf_coef=0.5, max_grad_norm=0.5, lam=0.95,
             nminibatches=4, noptepochs=4, cliprange=0.1, cliprange_vf=None,
             verbose=0, tensorboard_log="./11_11_full_logs_4/", _init_setup_model=True,
             policy_kwargs=None, full_tensorboard_log=False)


model.learn(total_timesteps=2000000,callback=None, seed=None,
            log_interval=1, tb_log_name="Full Logs", reset_num_timesteps=True)


model.save("ppo2_agent")


Here the model is saved after the 2 million steps. Is it possible to save the models in between?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.