1
$\begingroup$

I'm study and try a reinforcement learning. now im using gym and stable-baseline.

My project have a step where I calculate a reward with a function.

-> step()
-> calculate_reward()
-> return calculated reward
-> in step(): I set total_reward = -1000 and Done = True

But agent will interrupt everytime to this step becouse it using the same action.

Tick: 23
Price: 131.067600 - Next Price: 126.127800 - Action: 0 - Reward: 4.939800
Tick: 24
Price: 128.367200 - Next Price: 130.804100 - Action: 0 - Reward: -2.436900
Tick: 23
Price: 131.067600 - Next Price: 126.127800 - Action: 0 - Reward: 4.939800
Tick: 24
Price: 128.367200 - Next Price: 130.804100 - Action: 0 - Reward: -2.436900
Tick: 23
Price: 131.067600 - Next Price: 126.127800 - Action: 0 - Reward: 4.939800
Tick: 24
Price: 128.367200 - Next Price: 130.804100 - Action: 0 - Reward: -2.436900

What is wrong in my concept? My notebook colab is: https://colab.research.google.com/drive/1HxjONW_QvYPagk7clUrUPFAkpJ59OwZ8?usp=sharing

$\endgroup$
1
$\begingroup$

You could change some parameters of your A2C model, perhaps the learning rate or the alpha/epsilon/gamma/momentum, for instance:

model = A2C('MlpLstmPolicy', env, verbose=1, learning_rate=0.0001, alpha=0.001, momentum = 0.02)

or using a different LSTM architecture, for instance:

policy_kwargs = dict(net_arch=[64, 'lstm', dict(vf=[128, 128, 128], pi=[64, 64])])
model = A2C('MlpLstmPolicy', env, verbose=1,policy_kwargs = policy_kwargs)

I don't know which parameter could be the best, it depends on your data.

Note that stock markets might need data scaling techniques in order to reach better results.

$\endgroup$
3
  • $\begingroup$ Thank you. I've added a notebook $\endgroup$
    – Roger AI
    Sep 1 at 11:47
  • $\begingroup$ Great. I've answered accordingly. $\endgroup$ Sep 1 at 14:32
  • $\begingroup$ Well, now work up to ticker: 27, and the agent use everytime action 1, that is wrong becouse reward is negative. (pastebin.com/heciS4ZY) $\endgroup$
    – Roger AI
    Sep 2 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.