Questions tagged [openai-gym]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
1 answer
109 views

How can someone evaluate llama index model?

I have built a openAI llama index based model which takes multiple pdf and able to give chatbot based response. I want to evaluate the llm for accuracy. I already know method such as Rouge and Bleu. ...
Prashant Pandya's user avatar
0 votes
0 answers
28 views

State and next state is the same on openai gym atari

i am using OpenGym atari to trainning my Pacman agent, this is part of my code ...
tuyenhx's user avatar
0 votes
0 answers
33 views

find q-table for discrete action space

I am trying to use q-learning for a discrete observation space that is represented by: buffer: list of 200 integer values in [0,10] discard_counter: list of 200 integer values in [0, 4] capacity: ...
Jannis's user avatar
  • 1
0 votes
0 answers
14 views

Llimit/masking the space of actions to the legal ones only: A -> A(S)

I have difficulties on coding a way to limit the available actions as a function of the current agent state S. I am trying to use https://keras.io/examples/rl/ppo_cartpole/ (that works quite fine) and ...
fede72bari's user avatar
0 votes
1 answer
309 views

Compatibility of anytrading Gym environment with TF-Agents

All Gym/Gymnasium standard environments are compatible with TwnsorFlow RL agents, but when I tried to use TF-Agents with anytrading I get errors because some required methods and attributes seem to be ...
fede72bari's user avatar
0 votes
0 answers
108 views

ModuleNotFoundError: No module named 'gym_anytrading'

Windows 10 operating system, Anaconda used. import sys !conda install --yes --prefix {sys.prefix} -c anaconda gymnasium was successfully completed as well as <...
fede72bari's user avatar
0 votes
0 answers
44 views

Reinforcement learning with Q-learning doesn't seem to be learning

I'm learning reinforcement learning with Q-learning and I made a training script for Flappy Bird. The problem is that it doesn't seem to be learning and I'm not sure why. My guess is that there's ...
Be nice to others's user avatar
0 votes
1 answer
193 views

OpenAI Gym/Gymnasium Custom Env: How should done signal be defined in a continuous task/infinite horizon problem?

I am creating a custom gym environment that should abstract a non-episodic/continuous task. Generally gym requires to return ...
94621's user avatar
  • 1
1 vote
1 answer
42 views

Why use sampling instead of the mean value for policy in Reinforcement Learning?

I'm quite new in RL and I'm currently following David Silver's course on RL. But at the same time, I also want to get hands-on, so I followed this tutorial from Gymnasium documentation: https://...
GA Faza's user avatar
  • 13
0 votes
0 answers
18 views

Reinforcement Learning - What environment and algorithms should I use?

I have to do a project on Reinforcement Learning. Environment First, I need to choose an environment to use. It should meet one of two assumptions: it should be stochastic OR it should require ...
nietoperz21's user avatar
0 votes
0 answers
16 views

State space for dual deep q-network Reinforcement Learning Trading

how should I define state space for a ddqn trading agent, environment has observation space prev 99 candles transformed each into an gramian angular field image, prev 20 close prices difference, ...
user9085964's user avatar
0 votes
0 answers
31 views

How does a RL algorithm perceives OHLC data?

I'm creating a custom environment for RL trading. I created a dict action space composed by 1 box and 2 discrete variables. I was thinking about "rendering" the ohlc data in the box, but ...
user9085964's user avatar
0 votes
0 answers
724 views

how to define a multidiscrete action space

I want to define a Multi Discrete action space for Reinforcement Learning, gym style and Stable Baselines compatible (A2C, PPO) with the following structure : ...
user9085964's user avatar
0 votes
0 answers
250 views

How to build stock portfolio using deep reinforcement learning in Python by taking care of indicators for each given stock?

I have this Python code from this tutorial which is trading a stock (e.g. GME but it can be any) by taking care of its indicators ('SMA', 'RSI', 'OBV'). ...
YoYoYo's user avatar
  • 101
0 votes
1 answer
1k views

How to create custom action space in openai.gym

I am trying to upgrade code for custom environment written in gym==0.18.0 to latest version of gym. My current action space and observation space are defined as self.observation_space = np.ndarray(...
kumar's user avatar
  • 79
1 vote
1 answer
180 views

OpenAI Gym: gym.make() does not refer to updated Env code

I am building my a custom Gym environment and so far everything worked well following the guides spread all over the internet. However, I am now in a phase when frequent changes to the environment ...
PWillms's user avatar
  • 61
0 votes
1 answer
722 views

The observation returned by the `reset()` method is not contained with the observation space

i'm strugling to understand how custom observation_space should be coded and how there isn't so much info about some topics i have to ask here if for example i have to return a observation with an ...
Archlinux Developer's user avatar
2 votes
1 answer
134 views

How to Form the Training Examples for Deep Q Network in Reinforcement Learning?

Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
Della's user avatar
  • 315
0 votes
1 answer
897 views

gym car racing v0 using DQN

I am currently learning reinforcement learning and wanted to use it on the car racing-v0 environment. I have successfully made it using PPO algorithm and now I want to use a DQN algorithm but when I ...
Din's user avatar
  • 1
0 votes
0 answers
155 views

Reinforcement learning in real world without OpenAI Gym

I have been searching for reinforcement learning libraries or examples that aren't using a simulated environment like OpenAI Gym. I havn't been successful at all until yet and I would really ...
d1aren's user avatar
  • 9
0 votes
0 answers
597 views

Error:DQN expects a model that has one dimension for each action, in this case (1, 2, 1, 0)

I am building an RL agent for which the model is defined: ...
Michele Raso's user avatar
1 vote
2 answers
2k views

How to install custom pakages on Google-Colaboratory

I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'. I done the following things. First I cloned the git hub repository ...
zoraiz ali's user avatar
0 votes
2 answers
2k views

Understanding action space in stable baselines

I was trying to write reinforcement learning agent using stable-baselines3 library. The agent(abservations) method should return action. I went through different ...
Rnj's user avatar
  • 205
2 votes
1 answer
239 views

OpenAI Gym equivalent for supervised and/or unsupervised learning

OpenAI Gym has really normalized the way reinforcement learning is performed. It makes it possible for data scientists to separate model development and environment setup/building and to focus on what ...
Ali Hassaine's user avatar
3 votes
2 answers
486 views

How exactly does DQN learn?

I created my custom environment in gym, which is a maze. I use a DQN model with ...
Marci's user avatar
  • 31
1 vote
1 answer
366 views

Effects of slipperiness in OpenAI FrozenLake Environment

I am trying to wrap my head around the effects of is_slippery in the open.ai FrozenLake-v0 environment. From my results when ...
yudhiesh's user avatar
  • 213
0 votes
1 answer
91 views

Why MADDPG rather than taking all cooperating agents as a single meta-agent?

Since MADDPG uses a centralized critic for training, why not simply treat all cooperating agents as a single meta-agent with a concatenated observation space and a concatenated action space? In my ...
Forrest Wei's user avatar
1 vote
0 answers
253 views

Different probability distributions for each number of a MultiDiscrete action space

I have made a custom gym environment, and I have a question regarding the actions. I use a MultiDiscrete action space, that is, it provides a list of integer ...
Michail's user avatar
  • 11
2 votes
1 answer
1k views

What does anneal mean in the context of machine learning?

An article released by Open AI gives an overview of how Open AI Five works. There is a paragraph in the article stating: Our agent is trained to maximize the exponentially decayed sum of future ...
Reuben Walker's user avatar
0 votes
2 answers
2k views

No registered env with id: BanditTenArmedGaussian-v0 for the package gym_bandits of OpenAI

I've made the following instruction : git clone https://github.com/JKCooper2/gym-bandits.git cd gym-bandits pip install -e . After installing : ...
CechMS's user avatar
  • 61
1 vote
2 answers
3k views

What is the purpose of reward threshold in OpenAI Gym?

I've seen that OpenAI Gym environments can be registered with an optional reward threshold (reward_threshold) which represents: The reward threshold before the ...
Cristian M's user avatar
1 vote
0 answers
626 views

Does OpenAI Gym or Tensorforce require a normalized action space?

I am learning to use OpenAI Gym to make a custom environment with continuous action and observation spaces and apply reinforcement learning algorithms using the Tensorforce library. The problem is ...
Cristian M's user avatar
4 votes
1 answer
10k views

How to define discrete action space with continuous values in OpenAI Gym?

I am trying to use a reinforcement learning solution in an OpenAI Gym environment that has 6 discrete actions with continuous values, e.g. increase parameter 1 with 2.2, decrease parameter 1 with 1.6, ...
Cristian M's user avatar
1 vote
0 answers
196 views

Definition of obstacles in new OpenAI gym environment

I study a Reinforcement Learning algorithm that navigate an agent from one initial point to another in a complex environment where other agents and obstacles exists too. I want to make my own gym ...
dani's user avatar
  • 11
1 vote
0 answers
77 views

Sudden drop of score in the last few episodes

I was following this tutorial about lunar lander and deep Q learning with Tensorflow 2 and I noticed something odd. The problem was actually solved at episode 476 but then the score went from 259.90 ...
Marc's user avatar
  • 123
2 votes
1 answer
96 views

How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?

There's a popular solution to the CartPole game using Keras and Deep Q-Learning: https://keon.github.io/deep-q-learning/ But there's a line of code that's confusing, this same question has been asked ...
David Goudet's user avatar
3 votes
1 answer
3k views

Valid actions in OpenAI Gym

Why don't the gym environments come with "valid actions"? The normal gym environment accepts as input any action, even if it's not even possible. Is this a normal thing in reinforcement learning? Do ...
Muppet's user avatar
  • 787
3 votes
0 answers
157 views

Learn large, variable-size action space for Diplomacy game

I am making an environment using OpenAI gym for Diplomacy, and making an AI for it. In Diplomacy, a player has many units, and each unit has a number of moves available to it. Therefore, the player'...
Daniel Paczuski Bak's user avatar
1 vote
1 answer
63 views

Do RL agents learn the optimal "degree" of an action to take?

I have a game environment I want to train an RL model on. This environment has 2 fundamental actions that the agent can take; "Left" or "Right" (say, 0 or 1). However, the actions "Left" or "Right" ...
PyRsquared's user avatar
  • 1,604
1 vote
1 answer
289 views

Does a larger action space take longer to train an RL agent?

I am playing around with the openai gym to try and better understand reinforcement learning. One agent parameter you can modify is the action space i.e. the specific actions an agent can take in an ...
PyRsquared's user avatar
  • 1,604
11 votes
1 answer
4k views

Why could my DDQN get significantly worse after beating the game repeatedly?

I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...
Danny Tuppeny's user avatar
2 votes
2 answers
5k views

openai gym - what is an agent I can use with a multi-discrete action space?

I have a custom environment with a multi-discrete action space. The action and observation spaces are as follows: Action: ...
Daniel Paczuski Bak's user avatar
1 vote
1 answer
990 views

A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation

very good implementation of A2C continuous for Pendulum-v0 Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like: ...
mLstudent33's user avatar
1 vote
0 answers
140 views

CartPole v1 - Simple backprop with 1 hidden layer

I'm trying to solve the CartPole-v1 problem from OpenAI by using backprop on a one-layer neural network - while updating the model at every time step using State action values (Q(s,a)). I'm unable to ...
uncountably-infinite's user avatar
3 votes
0 answers
215 views

Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode

I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...
Sandeep Bhutani's user avatar
0 votes
1 answer
464 views

How does DQN solve Open AI Cartpole - v0?

Context I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with ...
Josiah L.'s user avatar
  • 103
2 votes
1 answer
6k views

PPO, A2C for continuous action spaces, math and code

Edit: Question has been edited to better reflect what I learned after asking the original question. I implemented the clipped objective PPO-clip as explained here: https://spinningup.openai.com/en/...
mLstudent33's user avatar
0 votes
1 answer
822 views

Openai Spaces for a modified environment

I have a 2-dimensional array of normalized data. I am using space = np.array([0,1,...366],[0,0.000001,.....1]) I need to fit this as an observation space in ...
Karthik Rajkumar's user avatar
0 votes
2 answers
554 views

Gym action space for board game with reward function

Im trying to design an openai gym environment that plays a quite simple board game where each player has 16 pieces that are exactly the same in regard to how they can move. The board is 10x10 and ...
Coronon's user avatar
  • 103
1 vote
1 answer
377 views

How to represent an image as state in a Q-table

I'm trying to do Q-learning with the Atari games using the gym python's package. I want to use the image as the state of my algorithm, but I came up with a doubt: ...
Neoares's user avatar
  • 153