Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

Should we sample z in VAE encoder during inference, when used in RL pipeline like World Models?

My question has been motivated by reading World Models by Ha and Schmidhuber. In shortcut, they introduce a RL framework where the current state (an image) is encoded via VAE into a latent vector $z$, ...
0
votes
0answers
6 views

In reinforcement learning, continue update weights after training?

Assuming an rl agent that has to be trained, and then you can mainain same weights over episodes: in a single episode, you have to firtly backup weights, udpate continuosly weights in the episodes, ...
0
votes
0answers
10 views

How to make fair comparison of multi-task RL models if I have unlimited test data?

The data comes from a simulator hence I have the possibility to generate unlimited data. The reward is 0 (no success) and success(1) if episode is successful. Now, the question is what metric to use ...
0
votes
0answers
22 views

Which is the best Machine Learning appoach/model when all features are not available for prediction?

I have 50 Features in a Dataset to predict 1 Variable "Units Sold". I am currently using XGBoost model (Supervised Learning) to train all these 50 Features and the accuracy of the model on ...
-1
votes
0answers
16 views

Deep q learning from scratch weights diverge to NaN

I'm trying to make a deep q learning algorithm with neural network from scratch, minbatch gradient descent, replay memory, and target network. But weights diverge to NaN after a around 40 episodes ...
1
vote
1answer
22 views

In DQN, why not use target network to predict current state Q values?

In DQN, why not use target network to predict current state Q values, and not only next state q values? In doing a basic dq learning algorithm with nn from scratch, with replay memory, and minibatch ...
0
votes
0answers
14 views

Reinforcement Learning in continuing tasks (non-episodic)

I'm looking for an algorithm for Deep Reinforcement Learning in non-episodic or continuing tasks. To be explicit, I'm looking for an algorithm that allows the agent to learn online without separate ...
1
vote
0answers
15 views

DQN CartPole-v1 neural network doesn't optimize

I'm doing my first dnq algorithm, I'm trying to build a dnq agent, and neural network from scratch, but it seems that neural network doesn't optimize, I did 2 hidden layers, with ReLU, and the output ...
0
votes
1answer
29 views

Bellman operator and contraction property

Currently, I am learning about Bellman Operator in Dynamic Programming and Reinforcement Learning. I would like to know why is Bellman operator contraction with respect to infinity norm? Why not ...
2
votes
1answer
64 views

How to make our own environments for reinforcement learning?

How can we make our own environments for reinforcement learning? I have a heavy doubt that a game engine is used.. And if a game engine is used, which game engine is used that can be downloaded for ...
0
votes
1answer
22 views

Which Algorithm did OpenAI used to create a hide and seek playing Agent?

I just saw this video on youtube: https://www.youtube.com/watch?v=kopoLzvh5jY&t=9s Which Algorithm did OpenAI used to create a hide and seek playing Agent? Was it Genetic Algorithm or Policy ...
0
votes
1answer
55 views

Actor Network Target Value in A2C Reinforcement Learning

In DQN, we use; $Target = r+\gamma v(s')$ equation to train (fit) our network. It is easy to understand since we use the $Target$ value as the dependent variable like we do in supervised learning. I....
0
votes
0answers
14 views

How to use binary numbers and plain floats in the same neural network input for reinforcement learning?

I'm modeling a system whose configuration can be represented by a binary array ([1,0,0,1] or [0,1,0,0], for example), and the agent can move on a 2D space (thus having 3 DoFs), and the action the ...
2
votes
1answer
25 views

Which Policy Gradient Method was used by Google's Deep Mind to teach AI to walk

I just saw this video on Youtube. Which Policy Gradient method was used to train the AI to walk? Was it DDPG or D4PG or what?
0
votes
1answer
38 views

Difference between $Q(s,a)$ ,$V^*(s)$ and $V^\pi(s)$ in Markov Decision Process?

I am new to RL and I am trying to understand how to find solutions of an MDP. This is what I understand so far -> since the nature of our environment is stochastic, at a state 's' if we take an ...
0
votes
1answer
11 views

Why is DDPG a Policy Gradient Method? [closed]

Why is DDPG a Policy Gradient Method even though it's actor does not output probability?
0
votes
0answers
39 views

Q values loss per episode and mean absolute error

I am new to deep reinforcement learning! I am following this code for my adaptation problem (doing actions) https://github.com/jaromiru/AI-blog/blob/master/CartPole-DQN.py I am wondering how I can ...
2
votes
2answers
88 views

Is reinforcement learning a subset of unsupervised learning?

According to this article: Reinforcement learning on the other hand, which is a subset of Unsupervised learning ... How true is this statement? Is there any scholarly discussion/writing on the ...
0
votes
1answer
13 views

How to build a comunication interface between a reinforcement learning and a videogame?

I want to try to build a reinforcement learning model in order to play old arcade videogame, I know about projects in order to build AI for playing videogames, but I don't know how could I build an ...
0
votes
1answer
30 views

Which policy gradient method is used for continuous action spaces?

Which policy gradient method is used that deals with continuous action spaces?
0
votes
0answers
15 views

Do all the Policy Gradient Methods work on the same principle/formula?

Do all the Policy Gradient Methods work based on the same formula? If Yes, What is that formula?
0
votes
0answers
13 views

What are some of the things that Policy Gradients Method can solve?

What are some of the things that Policy Gradients Method can solve that other Methods (like Q Learning and Genetic Algorithms) can't solve?
0
votes
0answers
21 views

What Math is required to learn Policy Gradients (Part of Reinforcement Learning)?

That seems a lot of Math... So, in order for me to understand it... What topics of math should I learn? Or to summarize: What is the math prerequisite to learn Policy Gradients?
0
votes
0answers
10 views

How to have a DNN output a classification for each user at once?

I have a Reinforcement Learning environment with an agent that allocates power values to different users. To do so, I have thought of implementing a deep neural network like the one shown in the ...
0
votes
0answers
6 views

How often to call DQN Replay memory?

I am noob to Machine Learning. I am running a DQN code, I understand the concept of Replay Memory but I don't understand how often shoud it be run. I saw one code calling it once in every 'episode' ...
0
votes
0answers
26 views

Understanding multi agent learning in OpenAI gym and stable-baselines

I was trying out developing multiagent reinforcement learning model using OpenAI stable baselines and gym as explained in this article. I am confused about how do we specify opponent agents. It seems ...
0
votes
0answers
9 views

What are the differences between DQN DDQN and Dueling Q network?

What are the differences between DQN DDQN and Dueling Q network? And how they expressed in the agent's behaviour? (For example in an Open AI Gym environment)
0
votes
0answers
11 views

How does implicit quantile network (IQN) keep the quantile value for a state action pair monotonous?

In my view, the implicit quantile network proposed by https://arxiv.org/pdf/1806.06923.pdf is used to estimate the quantile value of each state-action pair. Essentially the network is the approximated ...
1
vote
0answers
26 views

Deep Q-learning in non-episodic tasks

I want to use Deep Q-learning (specifically DDQL by Hasselt et al. 2015, but it is the same principle) in a non-episodic task (continuing). I know that it is possible to use Q-learning in continuing ...
1
vote
0answers
23 views

DQN DDQN and 3DQN differences?

I'm doing a course on reinforcement learning, and one of our tasks is to implement an agent on the Lunar lander continuous V2 environment from openAI gym. In order to solve the continuous problem, I ...
0
votes
2answers
47 views

Understanding action space in stable baselines

I was trying to write reinforcement learning agent using stable-baselines3 library. The agent(abservations) method should return action. I went through different ...
1
vote
0answers
7 views

Implementing DQN - Pylessons.com article - Queries [closed]

I have followed the below link to implement DQN Algorithm https://pylessons.com/CartPole-reinforcement-learning/ Can someone explain me: Why do we need to have else condition in act function during ...
1
vote
0answers
21 views

Reinforcement Learning reward is not converging to zero

I am new to reinforcement learning. I am trying to build an RL algorithm which will predict cloud hardware capacity required for an org in terms of compute, storage, memory. The algorithm which i have ...
0
votes
0answers
23 views

Blocked in my project with reinforcement learning.. any help?

I would like advice on a problem I'm trying to solve. I'm stuck... Here's an example: I have continuous data from a person such as what was eaten for a week (calories, carbohydrates, protein and fat) ...
0
votes
0answers
8 views

Forcing distinct, independent sets of outputs in a neural network architecture

I'm building a neural network architecture, which starts with an autoencoder, which reduces the dimensionality of the data and performs something akin to a PCA - finds a lower-dimensional ...
3
votes
2answers
351 views

How exactly does DQN learn?

I created my custom environment in gym, which is a maze. I use a DQN model with ...
1
vote
1answer
19 views

Quick question on basic Basic concept of experience replay

Due to my admitted newbie's understanding on the field, I'm about to ask a dummy question. While sampling batches, for example experience replay buffer which contains number of samples, after getting ...
0
votes
0answers
13 views

Notation for anti-optimal

What's the notation for the worst element/solution? The best "something" can be denoted with *, i.e.: $R^* = R_{best}$ $R^? = R_{worst}$
2
votes
1answer
60 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
0
votes
1answer
21 views

Reinforcement Learning model always gives different output

I am trying to build a reinforcement learning model for hardware capacity optimisation. The state of the model would input like CPU capacity utilisation, memory utilisation. The model is supposed to ...
3
votes
1answer
83 views

Confusion about the Bellman Equation

In some resources, the belman equation is shown as below: $v_{\pi}(s) = \sum\limits_{a}\pi(a|s)\sum\limits_{s',r}p(s',r|s,a)\big[r+\gamma v_{\pi}(s')\big] $ The thing that I confused is that, the $\pi$...
0
votes
0answers
97 views

How to define observation space for a custom environment in OpenAI gym?

I am a newbie in OpenAI gym. I need help to define the observation space for a sensor network environment. Suppose, there are N sensor nodes in a network. Each node has three features. So, I have ...
0
votes
1answer
16 views

Which solutions are there for RL agents when not all actions are always available?

I'm working in an RL environment where not all actions are always available. In this case, depending on the state where the environment is at, some of the actions are not available for the agent to ...
0
votes
0answers
11 views

How can my loss be stable while the gradient keeps growing?

I have been working on an Offline/Batch Reinforcement Learning problem where I am using a BCQ-DDQN model as a Q-table. The model input is a state of 8 dimensions, and the output is a vector of Q-...
1
vote
0answers
26 views

Is “nb_steps_warmup” set for each episode or globally?

When I configure a DQN agent, nb_steps_warmup can be set. Is this parameter set for each episode or once globally? What I am trying to ask is, imaging I have a game ...
0
votes
0answers
35 views

RL PPO Algorithm: Understanding the Value Function Loss term in PPO by OpenAI

In the Schulman 2017 PPO Paper, there is a value function loss term in the final loss in equation 9, where they state that the value function loss is the MSE of the target value and predicted value. ...
0
votes
1answer
36 views

How to report results of RL with high variance?

I run Q-learning and SARSA algortihms on the same problem but the results fluctuate heavily and when I draw them, there is no smooth graph. How should I repost the results? I run algorithms for 500 ...
1
vote
1answer
34 views

Effects of slipperiness in OpenAI FrozenLake Environment

I am trying to wrap my head around the effects of is_slippery in the open.ai FrozenLake-v0 environment. From my results when ...

1
2 3 4 5
11