8 votes
Accepted

Why could my DDQN get significantly worse after beating the game repeatedly?

This is called "catastrophic forgetting" and can be a serious problem in many RL scenarios. If you trained a neural network to recognise cats and dogs and did the following: Train it for many ...
user avatar
  • 27.3k
5 votes
Accepted

How to define discrete action space with continuous values in OpenAI Gym?

Here is a sample environment which demonstrates this. It relies on the environment to successfully filter out the correct continuous control element ...
user avatar
  • 251
4 votes

How exactly does DQN learn?

The way you have set your DQN up, it is designed to solve just one maze at a time. It has not (and cannot) learn to solve mazes in general, because it has no access to data about the layout of the ...
user avatar
  • 27.3k
3 votes
Accepted

Valid actions in OpenAI Gym

In general, if the agent is simply not able to take non-valid actions in a given environment (e.g. due to strict rules of a game, like chess), then it is standard practice to have the environment ...
user avatar
  • 27.3k
3 votes

What does anneal mean in the context of machine learning?

Annealing is short for simulated annealing. Simulated annealing is the process of slowly decreasing the probability of accepting worse solutions as the solution space is explored. Over the course of ...
user avatar
3 votes
Accepted

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

Again, the entropy equation coded was this: entropy = K.sum(0.5 * (K.log(2. * np.pi * sigma_sq) + 1.)) which looks different from what's given in the textbook photo above. They are the same after ...
user avatar
2 votes
Accepted

Card game for Gym: Reward shaping

My question is how to shape the rewards for card rejection and for winning a round. Any ideas? Positive or negative? In reinforcement learning, you must set rewards so that they are maximised when ...
user avatar
  • 27.3k
2 votes

What is a minimal setup to solve the CartPole-v0 with DQN?

As previously stated in the comment, you could simply look at the example in the repository you're using. These are a couple of comments on your choices: A rule of thumb is to decrease the number of ...
user avatar
2 votes

How exactly does DQN learn?

The only thing that you need to do is to start your agent and the goal/end at a random (non-overlapping) location. You can try your setup initially with an empty grid (no walls). If DQN learns, your ...
user avatar
  • 1,921
2 votes
Accepted

openai gym - what is an agent I can use with a multi-discrete action space?

OpenAI Baselines - or for me even better, Stable Baselines - has many model options which can handle MultiDicrete Action and/or Observation spaces. Building a custom gym environment is also quite ...
user avatar
2 votes
Accepted

How to Form the Training Examples for Deep Q Network in Reinforcement Learning?

But when training a neural network to represent the mapping, how exactly do I get the training samples? To make it more concrete, suppose the state $s\in\mathbb{R}^d$ is a $d$-dimensional vector and ...
user avatar
  • 27.3k
1 vote

How to install custom pakages on Google-Colaboratory

I think this issue is caused by the fact that your module has the same name as another package, which is the gym package for reinforcement learning environments. ...
user avatar
  • 5,766
1 vote

No registered env with id: BanditTenArmedGaussian-v0 for the package gym_bandits of OpenAI

The standard installation of OpenAI environments does not support Windows. See the GitHub issue here. To get that code to run, you'll have to switch another operating system (i.e., Linux or Mac) or ...
user avatar
1 vote

What is the purpose of reward threshold in OpenAI Gym?

I did some digging in the gym codebase, and at least as of v.0.18.0, gym itself doesn't appear to be using reward_threshold at all (as opposed to ...
user avatar
1 vote
Accepted

Do RL agents learn the optimal "degree" of an action to take?

If you are using a value-based method, like Q-learning in Deep Q Networks (DQN), then the "degree" concept has little meaning to the agent, and you are effectively training an agent to learn the best ...
user avatar
  • 27.3k
1 vote
Accepted

Does a larger action space take longer to train an RL agent?

I think there are two things in your question: The number of parameters of your network. So if you have more actions to predict the action layer of your network will have more parameters and it will ...
user avatar
  • 1,921
1 vote

openai gym - what is an agent I can use with a multi-discrete action space?

Suppose that right now your space is defined as follows n_actions = (10, 20, 30) action_space = MultiDiscrete(n_actions) A simple solution on the environment side ...
user avatar
1 vote
Accepted

How does DQN solve Open AI Cartpole - v0?

You are mixing up two concepts from reinforcement learning, reward and return (aka utility) Rewards are used to identify or specify goals of the agent. Whilst you can change them to help an agent ...
user avatar
  • 27.3k
1 vote
Accepted

PPO, A2C for continuous action spaces, math and code

Packt Publishing's "Deep Reinforcement Learning Hands-On" has an entire chapter on continuous action spaces. Here is the math in the book: and the code accompanying the book: code repo for book ...
user avatar
1 vote

Openai Spaces for a modified environment

In your case it seems you simply can return a 2D vector and extract out the components for that. You can take a look at the MountainCar example under classic control envs for a fully working case. But ...
user avatar
  • 11
1 vote
Accepted

What is wrong with this reinforcement learning environment ?

I can see two issues: Your environment is not tracking changes to state, just random success/fail based on self.initBlevel which is never modified to reflect ...
user avatar
  • 27.3k
1 vote

Why the invariant reward helps training?

Well, some inefficient agents will need more steps to reach the goal. Others will have a more target-oriented, efficient way to reach the goal. The efficient agents will need less steps and have a ...
user avatar
  • 241

Only top scored, non community-wiki answers of a minimum length are eligible