Questions tagged [openai-gym]
The openai-gym tag has no usage guidance.
55
questions
1
vote
1
answer
109
views
How can someone evaluate llama index model?
I have built a openAI llama index based model which takes multiple pdf and able to give chatbot based response. I want to evaluate the llm for accuracy. I already know method such as Rouge and Bleu. ...
0
votes
0
answers
28
views
State and next state is the same on openai gym atari
i am using OpenGym atari to trainning my Pacman agent, this is part of my code
...
- 1
0
votes
0
answers
33
views
find q-table for discrete action space
I am trying to use q-learning for a discrete observation space that is represented by:
buffer: list of 200 integer values in [0,10]
discard_counter: list of 200 integer values in [0, 4]
capacity: ...
- 1
0
votes
0
answers
14
views
Llimit/masking the space of actions to the legal ones only: A -> A(S)
I have difficulties on coding a way to limit the available actions as a function of the current agent state S. I am trying to use https://keras.io/examples/rl/ppo_cartpole/ (that works quite fine) and ...
- 61
0
votes
1
answer
309
views
Compatibility of anytrading Gym environment with TF-Agents
All Gym/Gymnasium standard environments are compatible with TwnsorFlow RL agents, but when I tried to use TF-Agents with anytrading I get errors because some required methods and attributes seem to be ...
- 61
0
votes
0
answers
108
views
ModuleNotFoundError: No module named 'gym_anytrading'
Windows 10 operating system, Anaconda used.
import sys
!conda install --yes --prefix {sys.prefix} -c anaconda gymnasium
was successfully completed as well as
<...
- 61
0
votes
0
answers
44
views
Reinforcement learning with Q-learning doesn't seem to be learning
I'm learning reinforcement learning with Q-learning and I made a training script for Flappy Bird. The problem is that it doesn't seem to be learning and I'm not sure why.
My guess is that there's ...
0
votes
1
answer
193
views
OpenAI Gym/Gymnasium Custom Env: How should done signal be defined in a continuous task/infinite horizon problem?
I am creating a custom gym environment that should abstract a non-episodic/continuous task.
Generally gym requires to return ...
- 1
1
vote
1
answer
42
views
Why use sampling instead of the mean value for policy in Reinforcement Learning?
I'm quite new in RL and I'm currently following David Silver's course on RL. But at the same time, I also want to get hands-on, so I followed this tutorial from Gymnasium documentation: https://...
- 13
0
votes
0
answers
18
views
Reinforcement Learning - What environment and algorithms should I use?
I have to do a project on Reinforcement Learning.
Environment
First, I need to choose an environment to use. It should meet one of two assumptions:
it should be stochastic OR
it should require ...
- 101
0
votes
0
answers
16
views
State space for dual deep q-network Reinforcement Learning Trading
how should I define state space for a ddqn trading agent, environment has observation space prev 99 candles transformed each into an gramian angular field image, prev 20 close prices difference, ...
0
votes
0
answers
31
views
How does a RL algorithm perceives OHLC data?
I'm creating a custom environment for RL trading.
I created a dict action space composed by 1 box and 2 discrete
variables.
I was thinking about "rendering" the ohlc data in the box, but ...
0
votes
0
answers
724
views
how to define a multidiscrete action space
I want to define a Multi Discrete action space for Reinforcement Learning, gym style and Stable Baselines compatible (A2C, PPO) with the following structure :
...
0
votes
0
answers
250
views
How to build stock portfolio using deep reinforcement learning in Python by taking care of indicators for each given stock?
I have this Python code from this tutorial which is trading a stock (e.g. GME but it can be any) by taking care of its indicators ('SMA', 'RSI', 'OBV').
...
- 101
0
votes
1
answer
1k
views
How to create custom action space in openai.gym
I am trying to upgrade code for custom environment written in gym==0.18.0 to latest version of gym. My current action space and observation space are defined as
self.observation_space = np.ndarray(...
- 79
1
vote
1
answer
180
views
OpenAI Gym: gym.make() does not refer to updated Env code
I am building my a custom Gym environment and so far everything worked well following the guides spread all over the internet. However, I am now in a phase when frequent changes to the environment ...
- 61
0
votes
1
answer
722
views
The observation returned by the `reset()` method is not contained with the observation space
i'm strugling to understand how custom observation_space should be coded and how there isn't so much info about some topics i have to ask here
if for example i have to return a observation with an ...
2
votes
1
answer
134
views
How to Form the Training Examples for Deep Q Network in Reinforcement Learning?
Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
- 315
0
votes
1
answer
897
views
gym car racing v0 using DQN
I am currently learning reinforcement learning and wanted to use it on the car racing-v0 environment. I have successfully made it using PPO algorithm and now I want to use a DQN algorithm but when I ...
- 1
0
votes
0
answers
155
views
Reinforcement learning in real world without OpenAI Gym
I have been searching for reinforcement learning libraries or examples that aren't using a simulated environment like OpenAI Gym. I havn't been successful at all until yet and I would really ...
- 9
0
votes
0
answers
597
views
Error:DQN expects a model that has one dimension for each action, in this case (1, 2, 1, 0)
I am building an RL agent for which the model is defined:
...
- 33
1
vote
2
answers
2k
views
How to install custom pakages on Google-Colaboratory
I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'.
I done the following things.
First I cloned the git hub repository ...
- 11
0
votes
2
answers
2k
views
Understanding action space in stable baselines
I was trying to write reinforcement learning agent using stable-baselines3 library. The agent(abservations) method should return action. I went through different ...
- 205
2
votes
1
answer
239
views
OpenAI Gym equivalent for supervised and/or unsupervised learning
OpenAI Gym has really normalized the way reinforcement learning is performed. It makes it possible for data scientists to separate model development and environment setup/building and to focus on what ...
- 129
3
votes
2
answers
486
views
How exactly does DQN learn?
I created my custom environment in gym, which is a maze. I use a DQN model with ...
- 31
1
vote
1
answer
366
views
Effects of slipperiness in OpenAI FrozenLake Environment
I am trying to wrap my head around the effects of is_slippery in the open.ai FrozenLake-v0 environment.
From my results when ...
- 213
0
votes
1
answer
91
views
Why MADDPG rather than taking all cooperating agents as a single meta-agent?
Since MADDPG uses a centralized critic for training, why not simply treat all cooperating agents as a single meta-agent with a concatenated observation space and a concatenated action space? In my ...
- 11
1
vote
0
answers
253
views
Different probability distributions for each number of a MultiDiscrete action space
I have made a custom gym environment, and I have a question regarding the actions. I use a MultiDiscrete action space, that is, it provides a list of integer ...
- 11
2
votes
1
answer
1k
views
What does anneal mean in the context of machine learning?
An article released by Open AI gives an overview of how Open AI Five works. There is a paragraph in the article stating:
Our agent is trained to maximize the exponentially decayed sum of future ...
0
votes
2
answers
2k
views
No registered env with id: BanditTenArmedGaussian-v0 for the package gym_bandits of OpenAI
I've made the following instruction :
git clone https://github.com/JKCooper2/gym-bandits.git
cd gym-bandits
pip install -e .
After installing :
...
- 61
1
vote
2
answers
3k
views
What is the purpose of reward threshold in OpenAI Gym?
I've seen that OpenAI Gym environments can be registered with an optional reward threshold (reward_threshold) which represents:
The reward threshold before the ...
- 177
1
vote
0
answers
626
views
Does OpenAI Gym or Tensorforce require a normalized action space?
I am learning to use OpenAI Gym to make a custom environment with continuous action and observation spaces and apply reinforcement learning algorithms using the Tensorforce library. The problem is ...
- 177
4
votes
1
answer
10k
views
How to define discrete action space with continuous values in OpenAI Gym?
I am trying to use a reinforcement learning solution in an OpenAI Gym environment that has 6 discrete actions with continuous values, e.g. increase parameter 1 with 2.2, decrease parameter 1 with 1.6, ...
- 177
1
vote
0
answers
196
views
Definition of obstacles in new OpenAI gym environment
I study a Reinforcement Learning algorithm that navigate an agent from one initial point to another in a complex environment where other agents and obstacles exists too.
I want to make my own gym ...
- 11
1
vote
0
answers
77
views
Sudden drop of score in the last few episodes
I was following this tutorial about lunar lander and deep Q learning with Tensorflow 2 and I noticed something odd.
The problem was actually solved at episode 476 but then the score went from 259.90 ...
- 123
2
votes
1
answer
96
views
How is the target_f updated in the Keras solution to the Deep Q-learning Cartpole/Gym algorithm?
There's a popular solution to the CartPole game using Keras and Deep Q-Learning:
https://keon.github.io/deep-q-learning/
But there's a line of code that's confusing, this same question has been asked ...
- 23
3
votes
1
answer
3k
views
Valid actions in OpenAI Gym
Why don't the gym environments come with "valid actions"? The normal gym environment accepts as input any action, even if it's not even possible.
Is this a normal thing in reinforcement learning? Do ...
- 787
3
votes
0
answers
157
views
Learn large, variable-size action space for Diplomacy game
I am making an environment using OpenAI gym for Diplomacy, and making an AI for it.
In Diplomacy, a player has many units, and each unit has a number of moves available to it.
Therefore, the player'...
1
vote
1
answer
63
views
Do RL agents learn the optimal "degree" of an action to take?
I have a game environment I want to train an RL model on. This environment has 2 fundamental actions that the agent can take; "Left" or "Right" (say, 0 or 1).
However, the actions "Left" or "Right" ...
- 1,604
1
vote
1
answer
289
views
Does a larger action space take longer to train an RL agent?
I am playing around with the openai gym to try and better understand reinforcement learning. One agent parameter you can modify is the action space i.e. the specific actions an agent can take in an ...
- 1,604
11
votes
1
answer
4k
views
Why could my DDQN get significantly worse after beating the game repeatedly?
I've been trying to train a DDQN to play OpenAI Gym's CartPole-v1, but found that although it starts off well and starts getting full score (500) repeatedly (at around 600 episodes in the pic below), ...
- 213
2
votes
2
answers
5k
views
openai gym - what is an agent I can use with a multi-discrete action space?
I have a custom environment with a multi-discrete action space.
The action and observation spaces are as follows:
Action:
...
1
vote
1
answer
990
views
A2C Continuous for Pendulum-v0 working implementation, negation for loss and entropy calculation
very good implementation of A2C continuous for Pendulum-v0
Code has snippet to stop execution when mean of last 10 or 20 is higher than -20 but the results look like:
...
- 574
1
vote
0
answers
140
views
CartPole v1 - Simple backprop with 1 hidden layer
I'm trying to solve the CartPole-v1 problem from OpenAI by using backprop on a one-layer neural network - while updating the model at every time step using State action values (Q(s,a)). I'm unable to ...
3
votes
0
answers
215
views
Reinforcement Learning using PPO2 in openai gym retro, mario not learning the clear the easy episode
I am training mario game in retro using ppo2 baselines for some time. I have tried level3 and level1 too. But even after full training when I play using saved checkpoints, the mario is not able to ...
- 894
0
votes
1
answer
464
views
How does DQN solve Open AI Cartpole - v0?
Context
I am confused about how a DQN is supposed to solve the cart pole problem since the rewards are so dense. I have been using pytorch example. I am aware of some solutions, but I have issue with ...
- 103
2
votes
1
answer
6k
views
PPO, A2C for continuous action spaces, math and code
Edit:
Question has been edited to better reflect what I learned after asking the original question.
I implemented the clipped objective PPO-clip as explained here: https://spinningup.openai.com/en/...
- 574
0
votes
1
answer
822
views
Openai Spaces for a modified environment
I have a 2-dimensional array of normalized data. I am using
space = np.array([0,1,...366],[0,0.000001,.....1])
I need to fit this as an observation space in ...
0
votes
2
answers
554
views
Gym action space for board game with reward function
Im trying to design an openai gym environment that plays a quite simple board game where each player has 16 pieces that are exactly the same in regard to how they can move.
The board is 10x10 and ...
- 103
1
vote
1
answer
377
views
How to represent an image as state in a Q-table
I'm trying to do Q-learning with the Atari games using the gym python's package.
I want to use the image as the state of my algorithm, but I came up with a doubt: ...
- 153