Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0
votes
1answer
37 views

Is there a mistake in Lecture 5 of Stanford CS234 available on youtube?

https://www.youtube.com/watch?v=buptHUzDKcE&list=PLoROMvodv4rOSOPzutgyCTapiGlY2Nd8u&index=5 At 53:45 Professor starts to describe temporal difference for linear value function approximation. ...
0
votes
0answers
22 views

How to use Tensorflow in Project Malmo

I've taken a tensorflow for beginner course and I'm wondering if Project Malmo can be use with Tensorflow, just a simple demonstration will do. Specifically, I'm trying to modify tutorial_6 of python-...
0
votes
0answers
18 views

Off-Policy Estimation - Importance Sampling with Negative Rewards

Importance sampling is a common method for calculating off-policy estimates in RL. I have been reading through some of the original documentation (D.G. Horvitz and D.J. Thompson, Powell, M.J. and ...
1
vote
1answer
34 views

Agent Collapse / Overfitting during Training

I'm new to reinforcement learning so please bear with me. I'm training an agent to play ms-Pacman using the actor-critic method. Below are the results of a couple of runs, in both graphs the orange ...
1
vote
2answers
42 views

Machine learning goal: given a population of 100,000 students, predict a group of 3,000, and minimize the median grade of that group

In other words, I am looking to predict students that will fail out of school before it happens. The data includes socioeconomic status and other related variables. I have tried an XGB binary ...
0
votes
0answers
14 views

Prioritized Experience Replay - which version is correct?

After reading a lot of stuff, I'm still not sure how to calculate the priorities for Prioritized Experience Replay (PER). Example code taken from here ...
0
votes
0answers
7 views

Wich activation function for DQL

After many research, I still can't find a neat answer about this question: When I found the loss of my state-action pair. I'm only backpropagating that loss true the network and setting all other ...
0
votes
0answers
20 views

Debugging Reinforcement Learning Model

I'm new to RL and I'm attempting to train an RL agent to play MsPacman in PyTorch. I've adapted the code from this tutorial on the PyTorch page for my problem. The DQN has the following architecture: <...
1
vote
1answer
31 views

Why does categorical_crossentropy work in when input is not 1-hot encoded?

I'm going through lessons on the REINFORCE algorithm to solve Cartpole/Pong/etc (using AIGym) and every one uses categorical_crossentropy as the loss function. What's confusing me is that ...
0
votes
1answer
25 views

Is this a valid stability concern/improvement for DQN/DDQN reinforcement training?

As you all know, DQN or DDQN are known for "unstable training". Let's use the well known "CartPole". The agent has to balance the stick and gets a reward of +1 per frame. You can reach the 195 ...
0
votes
0answers
18 views

Relation between optimal value function and optimal action value function

In equation 3.17 of Sutton and Barto's book: $$q_*(s, a)=\mathbb{E}[R_{t+1} + \gamma v_*(S_{t+1}) \mid S_t = s, A_t = a]$$ $G_{t+1}$ here have been replaced with $v_*(S_{t+1})$, but no reason has ...
1
vote
0answers
13 views

how do deep Q network deal with varying input size?

I am conducting research with multiply agents in an environment. The main concept of my methodology is a centralized control system, which means we take the positions, as well as other information, of ...
0
votes
0answers
12 views

Should I update action value functions when there is no change?

Suppose there is a website and the decision-maker wants to recommend some products to each customer visiting the website. Customers visit the websites in which the time interval between two ...
0
votes
0answers
9 views

reinforcement learning with both dynamic and static states

I am wondering is there any algorithm accommodate both dynamic and static states? In my problem, some states change over time depending on the actions taken. Other states are static. I am aware that ...
0
votes
1answer
44 views

(RL Curiosity) - “Exploration by Random Network Distillation” - what's the benefit?

Curiosity-Driven learning motivates the agent to explore unseen states. It does it by rewarding the agent more when its expectation differs from the actual next state. (orange color on the diagram). ...
3
votes
2answers
63 views

Can a classifier be trained with reinforcement learning without access to single classification results?

Question: Can a classifier be trained with reinforcement learning without access to single classification results? I want to train a classifier using reinforcement learning. However, there is one big ...
0
votes
1answer
26 views

Reinforcement (Q) learning: does it learn while in production?

I have a question for which I could not find the answer to it: While training reinforcement learning (using DQN), I get a model for the best reward for the next action. Now, if I deploy this model (i....
1
vote
0answers
6 views

Caps_Net. searching for example and library to use

Which library is most recommended and easy to use?
1
vote
1answer
14 views

How do I build a DQN which selects the correct objects in an environment based on the environment state?

I have an environment with 4 objects in it. All of these objects can either be selected or not selected. So the actions taken by my DQN should look like - ...
1
vote
0answers
115 views

Pytorch XLA to solve the spawn problems in a Colab Env

As reference only, here is my code It seems that torch.multiprocessing.set_start_method("spawn") can't be used in an Colab Env. Only 'fork' is allowed. I have ...
1
vote
1answer
11 views

Parameter initialization in a genetic algorithm

I'm using a neural network in a genetic algorithm. The neural network has 4 inputs (values between 0 and 1) and 4 outputs, corresponding to the probabilities of different actions. The neural network ...
3
votes
0answers
36 views

NN training with repetitive features

I posted the question also on ai.stackexchange but it didn't get any answers so I though I could try here. Here is a copy paste: Let's say you are training a NN in a RL setting where the state (i.e. ...
8
votes
1answer
121 views

Which ML approach to choose for the game AI when rewards are delayed?

Question: Which Machine Learning approach should I choose for the AI of my computer game, where the actions of the AI do not lead to immediate rewards, but delayed rewards instead? About me: I am a ...
4
votes
2answers
258 views

How much features are needed for Reinforcement learning?

I am trying to learn and use reinforcement learning. Now I have only 6 numeric features in my dataset. Can I still use RL? in other words can be using RL for a such a number of features sensible?
0
votes
1answer
19 views

Policy Gradient not “learning”

I'm attempting to implement the policy gradient taken from the "Hands-On Machine Learning" book by Geron, which can be found here. The notebook uses Tensorflow and I'm attempting to do it with PyTorch....
1
vote
0answers
61 views

Is this an implementation of Reinforcement learning?

I've written code in Python to replicate the results of a brief and simple paper about reinforcement learning. A brief description of the problem: we have a generated time series of returns (that ...
0
votes
0answers
24 views

First Simple DQN not learning to navigate maze

So I am currently attempting to write my first DQN implementation, where the aim is for the agent to learn to navigate the board from the top left to the bottom right while avoiding the hole right in ...
0
votes
0answers
13 views

Dueling DQN gradient with respect to a fully connected layer

Pertaining to this post, Dueling Network gradient with respect to Advantage stream, if Advantage and Value stream both obtain their values from a layer supposedly called A, such that Advantage = alpha....
2
votes
4answers
204 views

How to deal with class imbalance in a neural network?

Suppose we have a game and its action space contains two possible actions: A and B. We have a labelled dataset of state-action ...
0
votes
1answer
155 views

Result of Reinforcement learning

I've started reading some literature about reinforcement learning and I can't understand what is the result of the application of RL. I'll be more specific: let's have a time series problem in ...
1
vote
1answer
32 views

How to use a RBF kernel to create a “Kernel Space” using the similarity of each pair of point?

I am trying to use Semi-Unsupervised clustering using reinforcement learning following this paper. Assume I have n data-points each of which has d dimensions. I also have c pairwise constraints of ...
0
votes
0answers
9 views

Could you please give me an easy example in deep learning that explain Not-Differentiable?

A lot of paper said that their methods is end-to-end differentiable. (For example https://openreview.net/forum?id=SJxstlHFPH ) Also, a lot of reinforcement learning system said the differentiable ...
1
vote
0answers
12 views

Recommendation system with active learning

I have data where companies ask users to score a bunch of questions but some items may be randomly chosen while others are personalized. Users score higher in personalized questions on average. I have ...
0
votes
0answers
17 views

Can we use Q-Learning (or RL in general) for this problem?

Let's say that we have an algorithm that given a dataset point, it runs some analysis on it and returns the results. The algorithm has a user-defined parameter X that affects the run-time of the ...
0
votes
0answers
10 views

What is the difference between a convolutional neural network and reinforced learning in detecting MR anomalies?

I bumped into this paper on using deep learning to discover MRI lesions, using reinforcement learning to do so. I'm a novice in data science and thus far I've believed convolutional neural networks (...
0
votes
0answers
6 views

Identifying words of same category between different sets of text

I'm currently working on a PHP project, where I've received an interesting problem, to which I had the idea of using some sort of machine learning to solve. Problem: I have a table which needs to ...
0
votes
0answers
5 views

Separation line between solvable and insolvable cases in Multi-armed bandit

Consider the multi-armed bandits game with the following rules: We have 10 buckets each initialized as 2. Whenever you play n-th strategy on t-th step you add t to n-th bucket you add t to the content ...
0
votes
1answer
35 views

how to compute bernoulli entropy?

I am reading gail implementation code in openai baselines. they compute bernoulli entropy as one of the loss in adversary network loss function. In their code, they implement bernoulli entropy as ...
0
votes
0answers
31 views

How to render environment in Tensorforce?

How can one render the environment using the Tensorforce library? I've tried calling environment.render, but it says that the function does not exist. This is my ...
0
votes
0answers
24 views

What is the purpose of reward threshold in OpenAI Gym?

I've seen that OpenAI Gym environments can be registered with an optional reward threshold (reward_threshold) which represents: The reward threshold before the ...
1
vote
0answers
32 views

Does OpenAI Gym or Tensorforce require a normalized action space?

I am learning to use OpenAI Gym to make a custom environment with continuous action and observation spaces and apply reinforcement learning algorithms using the Tensorforce library. The problem is ...
0
votes
0answers
17 views

Pruning CNN filters & Reinforcement Learning

I don't know if this is an appropriate place to ask this type of questions but I don't know where to ask. I was reading this paper, it's about pruning cnn filters using reinforcement learning (policy ...
0
votes
0answers
5 views

Design reinforcement learning model to explore all optimal solutions?

I am working to use DQN and Policy Gradient reinforcement learning models to solve classic maze escaping problems. So far I ...
0
votes
0answers
13 views

Entropy applied to policy gradient prevent our agent from being stuck in the local minimum?

In the information theory, the entropy is a measure of uncertainty in some system. Being applied to agent policy, entropy shows how much the agent is uncertain about which action to make. In math ...
0
votes
0answers
9 views

Reward Function for a model-free MDP

I am trying to build a program which has to decide in a completely stochastic environment. So it has to be model-free and Q-learning is suitable for that. I just have one problem, my rewards are not ...
0
votes
0answers
8 views

For off policy reinforcement learning how different can the current policy be from the policies which generated the data

Say we have two policies and we use one to generate data with. We now want to use this data to optimize the second policy (the two policies are defined with the same input and output space but with ...
0
votes
0answers
5 views

Reward engineering to replace single terminal reward (exponential utility of terminal wealth)

My goal is to use reinforcement learning to train the agent (the trader) to maximize the exponential utility of his P&L (profit and loss) at a terminal time T. Therefore the natural formulation of ...
1
vote
0answers
14 views

Informed sender in Lazaridou et al. (2017) 's guessing game

I'm trying to implement the informed sender from Multi-Agent Cooperation and the Emergence of (Natural) Language (Lazaridou et al., 2017). However I'm confused about the tensor shape in the forward ...
0
votes
0answers
13 views

Intuition behind policy gradient

I like how Adrien Lucas explained intuitively the policy gradient. I would like to convince myself without using his code. Without the animation, how can I code the above recursive function using ...
2
votes
1answer
403 views

How to define discrete action space with continuous values in OpenAI Gym?

I am trying to use a reinforcement learning solution in an OpenAI Gym environment that has 6 discrete actions with continuous values, e.g. increase parameter 1 with 2.2, decrease parameter 1 with 1.6, ...

1
2 3 4 5
10