I am training an RL agent using PPO2 algorithm. Iam using stable-baselines library. During the training process, my rewards are slowly increasing and stabilizing, but are falling down suddenly. I figured its because of the catastrophic forgetting of the network. So,I would like to save my model when the mean episodic reward becomes maximum.
So here I would like to save the model when the training rewards are higer. My code
import pandas as pd from stable_baselines.common.policies import MlpPolicy from stable_baselines.common.policies import FeedForwardPolicy, register_policy from stable_baselines.common.vec_env import DummyVecEnv from stable_baselines import PPO2 from Env import Env import matplotlib.pyplot as plt from stable_baselines.logger import configure configure() # multiprocess environment n_cpu = 8 env = DummyVecEnv([lambda: AHUenv() for i in range(n_cpu)]) # Custom MLP policy of three layers of size 128 each class CustomPolicy(FeedForwardPolicy): def __init__(self, *args, **kwargs): super(AhuCustomPolicy, self).__init__(*args, **kwargs, net_arch=[dict(pi=[128, 128], vf=[128, 128])], feature_extraction="mlp") # Register the policy, it will check that the name is not already taken register_policy('CustomPolicy', CustomPolicy) model = PPO2(CustomPolicy,env,gamma=0.9, n_steps=88, ent_coef=0.01, learning_rate=2.5e-4, vf_coef=0.5, max_grad_norm=0.5, lam=0.95, nminibatches=4, noptepochs=4, cliprange=0.1, cliprange_vf=None, verbose=0, tensorboard_log="./11_11_full_logs_4/", _init_setup_model=True, policy_kwargs=None, full_tensorboard_log=False) model.learn(total_timesteps=2000000,callback=None, seed=None, log_interval=1, tb_log_name="Full Logs", reset_num_timesteps=True) model.save("ppo2_agent")
Here the model is saved after the 2 million steps. Is it possible to save the models in between?