The data comes from a simulator hence I have the possibility to generate unlimited data. The reward is 0 (no success) and success(1) if episode is successful. Now, the question is what metric to use for evaluation? For example, if I choose the average success rate as metric and hypothetically someone trains the RL model on one episode and the reward is 1 then the average will be 1. So I have to choose min. number of episodes. Should it be 10, 1000, 10000 and how do I calculate it from the data? Each episode is separate task so each episode might be different length than the previous one and might have different complexity.

Thank you.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.