Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
1
vote
0answers
36 views

In a double Deep Q network what would happen if we switch the roles of both networks

We normally use the online network for action selection and the target network for evaluation , would there be a difference if we switched the roles? Because in the case Of Double Q learning, we ...
1
vote
1answer
65 views

How is this score function estimator derived?

In this paper they have this equation, where they use the score function estimator, to estimate the gradient of an expectation. How did they derive this?
1
vote
0answers
24 views

How can I build a simulation environment that assess different risk policies? [closed]

I work in fin-tech and would like to build some sort of simulation program to assess how different inputs will impact net revenue. For example, if we create new policies based on ML scores, how would ...
0
votes
1answer
155 views

Monte Carlo for non-episodic tasks

In Sutton's textbook (Chapter 5) it says "To ensure that well-defined returns are available, here we define Monte Carlo methods only for episodic tasks". Can someone explain what exactly ...
0
votes
1answer
63 views

Is there a mistake in Lecture 5 of Stanford CS234 available on youtube?

https://www.youtube.com/watch?v=buptHUzDKcE&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&index=5 At 53:45 Professor starts to describe temporal difference for linear value function approximation. ...
1
vote
1answer
207 views

Agent Collapse / Overfitting during Training

I'm new to reinforcement learning so please bear with me. I'm training an agent to play ms-Pacman using the actor-critic method. Below are the results of a couple of runs, in both graphs the orange ...
1
vote
2answers
46 views

Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group

In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables. I have tried an XGB binary ...
1
vote
0answers
97 views

Prioritized Experience Replay - which version is correct?

After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER). Example code taken from here ...
1
vote
0answers
15 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...
1
vote
1answer
69 views

Why does categorical_crossentropy work in when input is not 1-hot encoded?

I'm going through lessons on the REINFORCE algorithm to solve Cartpole/Pong/etc (using AIGym) and every one uses categorical_crossentropy as the loss function. What's confusing me is that ...
0
votes
1answer
159 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
1
vote
0answers
88 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
1answer
113 views

(RL Curiosity) - "Exploration by Random Network Distillation" - what's the benefit?

Curiosity-Driven learning motivates the agent to explore unseen states. It does it by rewarding the agent more when its expectation differs from the actual next state. (orange color on the diagram). ...
3
votes
2answers
139 views

Can a classifier be trained with reinforcement learning without access to single classification results?

Question: Can a classifier be trained with reinforcement learning without access to single classification results? I want to train a classifier using reinforcement learning. However, there is one big ...
0
votes
1answer
39 views

Reinforcement (Q) learning: does it learn while in production?

I have a question for which I could not find the answer to it: While training reinforcement learning (using DQN), I get a model for the best reward for the next action. Now, if I deploy this model (i....
1
vote
0answers
7 views

Caps_Net. searching for example and library to use

Which library is most recommended and easy to use?
1
vote
1answer
82 views

How do I build a DQN which selects the correct objects in an environment based on the environment state?

I have an environment with 4 objects in it. All of these objects can either be selected or not selected. So the actions taken by my DQN should look like - ...
1
vote
0answers
205 views

Pytorch XLA to solve the spawn problems in a Colab Env

As reference only, here is my code It seems that torch.multiprocessing.set_start_method("spawn") can't be used in an Colab Env. Only 'fork' is allowed. I have ...
2
votes
1answer
75 views

Parameter initialization in a genetic algorithm

I'm using a neural network in a genetic algorithm. The neural network has 4 inputs (values between 0 and 1) and ...
3
votes
0answers
41 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
8
votes
1answer
151 views

Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
4
votes
2answers
394 views

How much features are needed for Reinforcement learning?

I am trying to learn and use reinforcement learning. Now I have only 6 numeric features in my dataset. Can I still use RL? in other words can be using RL for a such a number of features sensible?
2
votes
1answer
53 views

Policy Gradient not "learning"

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
1
vote
0answers
69 views

Is this an implementation of Reinforcement learning?

I've written code in Python to replicate the results of a brief and simple paper about reinforcement learning. A brief description of the problem: we have a generated time series of returns (that ...
2
votes
4answers
1k views

How to deal with class imbalance in a neural network?

Suppose we have a game and its action space contains two possible actions: A and B. We have a labelled dataset of state-action ...
1
vote
2answers
260 views

Result of Reinforcement learning

I've started reading some literature about reinforcement learning and I can't understand what is the result of the application of RL. I'll be more specific: let's have a time series problem in ...
1
vote
1answer
62 views

How to use a RBF kernel to create a "Kernel Space" using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
1
vote
1answer
40 views

Recommendation system with active learning

I have data where companies ask users to score a bunch of questions but some items may be randomly chosen while others are personalized. Users score higher in personalized questions on average. I have ...
1
vote
1answer
296 views

how to compute bernoulli entropy?

I am reading gail implementation code in openai baselines. they compute bernoulli entropy as one of the loss in adversary network loss function. In their code, they implement bernoulli entropy as ...
0
votes
1answer
302 views

How to render environment in Tensorforce?

How can one render the environment using the Tensorforce library? I've tried calling environment.render, but it says that the function does not exist. This is my ...
1
vote
2answers
894 views

What is the purpose of reward threshold in OpenAI Gym?

I've seen that OpenAI Gym environments can be registered with an optional reward threshold (reward_threshold) which represents: The reward threshold before the ...
1
vote
0answers
381 views

Does OpenAI Gym or Tensorforce require a normalized action space?

I am learning to use OpenAI Gym to make a custom environment with continuous action and observation spaces and apply reinforcement learning algorithms using the Tensorforce library. The problem is ...
1
vote
0answers
135 views

Entropy applied to policy gradient prevent our agent from being stuck in the local minimum?

In the information theory, the entropy is a measure of uncertainty in some system. Being applied to agent policy, entropy shows how much the agent is uncertain about which action to make. In math ...
1
vote
0answers
15 views

Informed sender in Lazaridou et al. (2017) 's guessing game

I'm trying to implement the informed sender from Multi-Agent Cooperation and the Emergence of (Natural) Language (Lazaridou et al., 2017). However I'm confused about the tensor shape in the forward ...
3
votes
1answer
6k views

How to define discrete action space with continuous values in OpenAI Gym?

I am trying to use a reinforcement learning solution in an OpenAI Gym environment that has 6 discrete actions with continuous values, e.g. increase parameter 1 with 2.2, decrease parameter 1 with 1.6, ...
0
votes
1answer
772 views

How to generate plot of reward and its variance?

I am new to reinforcement learning and I would like know how to generate a learning curve plot such as that shown below (taken from this blog post), that illustrates the reward (return) and its ...
1
vote
2answers
42 views

Free a bit of RAM space

Here is Colab Notebook After 1500 episodes if batch_size=256, the RAM crashed. With Colab, I have the equivalent of 25.5 gigs of RAM. Is it normal? Or I don't have ...
1
vote
0answers
284 views

How can I save this model(s) and why is the use for “with tf.Graph().as_default()”

I have been trying to train and then compile this RL algo. My problem comes when I want to save the three models. Here is how the neural networks are defined: ...
2
votes
0answers
37 views

Find highest reward for epsilon-greedy bandit program

I started to learn reinforcement learning, the first example is handling bandit program using epsilon-greedy method, In this example, there are three bandit machines used, the output is the mean value ...
1
vote
0answers
28 views

Action selection in actor-critic algorithm:

I have an action space that is just a list of values given by acts = [i for i in range(10, 100, 10)]. According to pytorch documentary, the loss is calculated as below. Could someone explain to me how ...
2
votes
1answer
197 views

Representation of state space, action space and reward system for RL porblem

I am trying to solve the problem of an agent dynamically discovering(start with no information about the environment) the environment and to explore as much of the environment as possible without ...
1
vote
1answer
23 views

The meaning of γt−t0 in Reinforcement learning with pytorch

When reading pytorch tutorial: Our aim will be to train a policy that tries to maximize the discounted, cumulative reward Rt0=∑∞t=t0γt−t0rt, where ...
1
vote
0answers
108 views

Definition of obstacles in new OpenAI gym environment

I study a Reinforcement Learning algorithm that navigate an agent from one initial point to another in a complex environment where other agents and obstacles exists too. I want to make my own gym ...
1
vote
0answers
75 views

Sudden drop of score in the last few episodes

I was following this tutorial about lunar lander and deep Q learning with Tensorflow 2 and I noticed something odd. The problem was actually solved at episode 476 but then the score went from 259.90 ...
2
votes
1answer
143 views

Purpose of trace-decay parameter in eligibility traces

In TD/SARSA-lambda, eligibility traces are decayed after each step by multiplying by the discount rate and the trace-decay parameter. I understand that: The discount rate is used to reduce the value ...
1
vote
0answers
101 views

Reward(t) vs. Reward(t+1) ? Reinforcement Learning, Q-learning

In "Reinforcement Learning, An Introduction; Richard S. Sutton and Andrew G. Barto" is written on page 70: 3.1 The Agent–Environment Interface At each time step t, the agent receives some ...
3
votes
1answer
43 views

What's the definition of retraining?

In transfer learning, we always use new data to retrain the pre-trained model. But, what is the specific and official definition of retraining? Or what papers mentioned this definition, in transfer ...
1
vote
0answers
97 views

Q-learning, state transition, immediate rewards (trading logic)

I've been thinking about how to correctly calculate rewards for several weeks now. Here is a grid example: ...
0
votes
1answer
346 views

When should the last action be included in the state in reinforcement learning?

I am having some confusion as to whether the action should be included as part of the state input to an agent in a reinforcement learning setting (state-action pair). As from my observation, this is ...
1
vote
0answers
19 views

Deep Reinforcement Learning [closed]

I am new in this field and I am trying to understand the Deep Deterministic Policy Gradient (DDPG) model. I can not understand the use of the target network in this model. Why do we need to define the ...

1 2 3
4
5
12