Questions tagged [openai-gym]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
2 votes
1 answer
31 views

How to Form the Training Examples for Deep Q Network in Reinforcement Learning?

Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
user avatar
  • 283
0 votes
0 answers
66 views

gym car racing v0 using DQN

I am currently learning reinforcement learning and wanted to use it on the car racing-v0 environment. I have successfully made it using PPO algorithm and now I want to use a DQN algorithm but when I ...
user avatar
  • 1
0 votes
0 answers
11 views

Chained Decisions in Reinforcement Learning

I am working on a project of portfolio optimization with reinforcement learning. I would like incorporate a dependent decision process: Decide which asset should be bought. Decide about the amount ...
user avatar
  • 3
0 votes
0 answers
13 views

Stack as many industrial components as possible in a crate

The exact problem is a crate of industrial parts, made by injection molding in very high quantities. The objective is to put as much parts as possible in one crate. This is done by a small robotic arm ...
user avatar
  • 111
1 vote
0 answers
35 views

Reinforcement learning in real world without OpenAI Gym

I have been searching for reinforcement learning libraries or examples that aren't using a simulated environment like OpenAI Gym. I havn't been successful at all until yet and I would really ...
user avatar
  • 11
0 votes
0 answers
233 views

Error:DQN expects a model that has one dimension for each action, in this case (1, 2, 1, 0)

I am building an RL agent for which the model is defined: ...
user avatar
1 vote
2 answers
233 views

How to install custom pakages on Google-Colaboratory

I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'. I done the following things. First I cloned the git hub repository ...
user avatar
0 votes
0 answers
19 views

build Environment using OpenAI Gym

I have an environment which contains 26 states. Each episode sample have an terminal state.I want my agent to learn how to get to the terminal state faster by using the minimum sequences of action. ...
user avatar
0 votes
0 answers
426 views

Package directory does not exist

I'm trying to install python packages in COLAB using the following setup.py file. ...
user avatar
0 votes
0 answers
6 views

Reinforcement Learning for distributing items on multiple places, i.e., scale balancing

I am interested in creating a reinforcement learning algorithm for assigning items to different buckets, such that the buckets are almost the same weight, i.e., scale but with more than two places to ...
user avatar
0 votes
0 answers
26 views

Reinforcement Learning End Effector Moving To Camera and Stops Learning

I am working on training a 3 finger Jaw gripper. The environment I setup is this: UR10 3 finger robot Pybullet for Simulation Stable baselines and DDPG Observation space is RGB image stacked with ...
user avatar
0 votes
2 answers
587 views

Understanding action space in stable baselines

I was trying to write reinforcement learning agent using stable-baselines3 library. The agent(abservations) method should return action. I went through different ...
user avatar
  • 165
2 votes
1 answer
57 views

OpenAI Gym equivalent for supervised and/or unsupervised learning

OpenAI Gym has really normalized the way reinforcement learning is performed. It makes it possible for data scientists to separate model development and environment setup/building and to focus on what ...
user avatar
3 votes
2 answers
423 views

How exactly does DQN learn?

I created my custom environment in gym, which is a maze. I use a DQN model with ...
user avatar
  • 31
1 vote
1 answer
61 views

Effects of slipperiness in OpenAI FrozenLake Environment

I am trying to wrap my head around the effects of is_slippery in the open.ai FrozenLake-v0 environment. From my results when ...
user avatar
  • 213
0 votes
1 answer
66 views

Why MADDPG rather than taking all cooperating agents as a single meta-agent?

Since MADDPG uses a centralized critic for training, why not simply treat all cooperating agents as a single meta-agent with a concatenated observation space and a concatenated action space? In my ...
user avatar
1 vote
0 answers
138 views

Different probability distributions for each number of a MultiDiscrete action space

I have made a custom gym environment, and I have a question regarding the actions. I use a MultiDiscrete action space, that is, it provides a list of integer ...
user avatar
  • 11
1 vote
1 answer
252 views

What does anneal mean in the context of machine learning?

An article released by Open AI gives an overview of how Open AI Five works. There is a paragraph in the article stating: Our agent is trained to maximize the exponentially decayed sum of future ...
user avatar
0 votes
1 answer
2k views

No registered env with id: BanditTenArmedGaussian-v0 for the package gym_bandits of OpenAI

I've made the following instruction : git clone https://github.com/JKCooper2/gym-bandits.git cd gym-bandits pip install -e . After installing : ...
user avatar
  • 61
1 vote
2 answers
1k views

What is the purpose of reward threshold in OpenAI Gym?

I've seen that OpenAI Gym environments can be registered with an optional reward threshold (reward_threshold) which represents: The reward threshold before the ...
user avatar
1 vote
0 answers
451 views

Does OpenAI Gym or Tensorforce require a normalized action space?

I am learning to use OpenAI Gym to make a custom environment with continuous action and observation spaces and apply reinforcement learning algorithms using the Tensorforce library. The problem is ...
user avatar
3 votes
1 answer
7k views

How to define discrete action space with continuous values in OpenAI Gym?

I am trying to use a reinforcement learning solution in an OpenAI Gym environment that has 6 discrete actions with continuous values, e.g. increase parameter 1 with 2.2, decrease parameter 1 with 1.6, ...
user avatar
1 vote
0 answers
116 views

Definition of obstacles in new OpenAI gym environment

I study a Reinforcement Learning algorithm that navigate an agent from one initial point to another in a complex environment where other agents and obstacles exists too. I want to make my own gym ...
user avatar
  • 11
1 vote
0 answers
75 views

Sudden drop of score in the last few episodes

I was following this tutorial about lunar lander and deep Q learning with Tensorflow 2 and I noticed something odd. The problem was actually solved at episode 476 but then the score went from 259.90 ...
user avatar
  • 113
1 vote
1 answer
51 views

How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?

There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/ But there's a line of code that's confusing, this same question has been asked ...
user avatar
3 votes
1 answer
2k views

Valid actions in OpenAI Gym

Why don't the gym environments come with "valid actions"? The normal gym environment accepts as input any action, even if it's not even possible. Is this a normal thing in reinforcement learning? Do ...
user avatar
  • 677
3 votes
0 answers
149 views

Learn large, variable-size action space for Diplomacy game

I am making an environment using OpenAI gym for Diplomacy, and making an AI for it. In Diplomacy, a player has many units, and each unit has a number of moves available to it. Therefore, the player'...
user avatar
1 vote
1 answer
50 views

Do RL agents learn the optimal "degree" of an action to take?

I have a game environment I want to train an RL model on. This environment has 2 fundamental actions that the agent can take; "Left" or "Right" (say, 0 or 1). However, the actions "Left" or "Right" ...
user avatar
  • 1,484
1 vote
1 answer
156 views

Does a larger action space take longer to train an RL agent?

I am playing around with the openai gym to try and better understand reinforcement learning. One agent parameter you can modify is the action space i.e. the specific actions an agent can take in an ...
user avatar
  • 1,484
8 votes
1 answer
2k views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...
user avatar
2 votes
2 answers
3k views

openai gym - what is an agent I can use with a multi-discrete action space?

I have a custom environment with a multi-discrete action space. The action and observation spaces are as follows: Action: ...
user avatar
1 vote
1 answer
820 views

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

very good implementation of A2C continuous for Pendulum-v0 Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like: ...
user avatar
1 vote
0 answers
99 views

CartPole v1 - Simple backprop with 1 hidden layer

I'm trying to solve the CartPole-v1 problem from OpenAI by using backprop on a one-layer neural network - while updating the model at every time step using State action values (Q(s,a)). I'm unable to ...
user avatar
3 votes
0 answers
169 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...
user avatar
0 votes
1 answer
412 views

How does DQN solve Open AI Cartpole - v0?

Context I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with ...
user avatar
  • 103
2 votes
1 answer
5k views

PPO, A2C for continuous action spaces, math and code

Edit: Question has been edited to better reflect what I learned after asking the original question. I implemented the clipped objective PPO-clip as explained here: https://spinningup.openai.com/en/...
user avatar
0 votes
1 answer
507 views

Openai Spaces for a modified environment

I have a 2-dimensional array of normalized data. I am using space = np.array([0,1,...366],[0,0.000001,.....1]) I need to fit this as an observation space in ...
user avatar
0 votes
2 answers
406 views

Gym action space for board game with reward function

Im trying to design an openai gym environment that plays a quite simple board game where each player has 16 pieces that are exactly the same in regard to how they can move. The board is 10x10 and ...
user avatar
  • 103
1 vote
1 answer
215 views

How to represent an image as state in a Q-table

I'm trying to do Q-learning with the Atari games using the gym python's package. I want to use the image as the state of my algorithm, but I came up with a doubt: ...
user avatar
  • 153
1 vote
0 answers
266 views

comparison of linear Q-learning and DQN

I saw in DQN nature paper 2015 https://www.nature.com/articles/nature14236(Extended Data Table 4) some comparisons between DQN and linear Q-learning. The ratio ...
user avatar
  • 1,526
3 votes
1 answer
677 views

What is wrong with this reinforcement learning environment ?

I'm working on below reinforcement learning problem: I have bottle of fix capacity (say 5 liters). At the bottom of bottle there is cock to remove water. The distribution of removal of water is not ...
user avatar
1 vote
1 answer
38 views

Why the invariant reward helps training?

I am new to Machine Learning, and I am trying to solve MountainCar-v0 using Q-learning. I can solve the problem now, but I am still confused. According to the MountainCar-v0's Wiki, the reward ...
user avatar
1 vote
1 answer
622 views

Card game for Gym: Reward shaping

I am working on a card game for openai gym and currently I ask myself how to shape the reward function for it. One round of the game consists of each player picking a card from its hand, whereas not ...
user avatar
  • 113
4 votes
1 answer
2k views

What is a minimal setup to solve the CartPole-v0 with DQN?

I solved the CartPole-v0 with a CEM agent pretty easily (experiments and code), but I struggle to find a setup which works with DQN. Do you know which parameters should be adjusted so that the mean ...
user avatar
  • 17.4k