Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0 votes
0 answers
3 views

Optimizing utility and square root of divergence via SGD

I am trying to optimize a objective for learning-to-rank, which tries to max(min) the utility(risk) of a ranking function with logged user feedback data. The idea is to learn a ranking policy which ...
user avatar
0 votes
0 answers
18 views

Data Imbalance in Contextual Bandit with Thompson Sampling

I'm working with the Online Logistic Regression Algorithm (Algorithm 3) of Chapelle and Li in their paper, "An Empirical Evaluation of Thompson Sampling" (https://papers.nips.cc/paper/2011/...
user avatar
  • 1
0 votes
0 answers
43 views

Reinforcement learning using univalent and multivalent heterogeneous features

Problem introduction I have a game in which human players play levels (just like the famous casual game candy crush) where each level has its own properties and its own difficulty. In said game, the ...
user avatar
  • 1
0 votes
0 answers
13 views

MAML applied to unknown tasks in Meta-RL paper

I have been reviewing a paper on Meta-RL applied to Non-Stationary (NS) environment (Paper on arxiv), which assume that in a certain context of interest NS may be modeled as a switching environment ...
user avatar
  • 111
0 votes
0 answers
5 views

Can I train a model directly at onnx?

I'm trying to make an unreal engine game and I recently discovered that I can use onnx as my training framework, so I can load models at c++. I pretend to make reinforcement learning agents for my ...
user avatar
0 votes
0 answers
11 views

Why is the PPO agent in RL giving negative rewards after each iteration during the training process and what are the possible hyperparameter values?

I am using the mujoco simulator as my training environment. I loaded Ant-v3 for the agent to train on. It is persistently producing negative rewards after each iteration performed.
user avatar
  • 1
3 votes
2 answers
134 views

Manipulating noise to get some data in right format and apply it to task using PPO

Warning: I understand that my question may seem strange, stupid, and impossible, but let's just think about this interesting problem. I would not ask a question like: how to create an AGI in google ...
user avatar
0 votes
0 answers
12 views

What enables transformers or very deep models "plan" ahead for sequential decision making?

I was watching this amazing lecture by Oriol Vinyals. On one slide, there is a question asking if the very deep models plan. Transformer models or models employed in applications like Dialogue ...
user avatar
0 votes
0 answers
8 views

Computing probabilities in Plackett-Luce model

I am trying to implement a Plackett-Luce model for learning to rank from click data. Specifically, I am following the paper: Doubly-Robust Estimation for Correcting Position-Bias in Click Feedback for ...
user avatar
0 votes
0 answers
28 views

Can i have the input to a neural network be a set of 2d coordinates if i run them through a convolution layer? [duplicate]

I asked this question a few days ago with no response and still do not have an answer so I will ask again. I am training a reinforcement learning agent on a 2D grid. It is fed in its position, and the ...
user avatar
0 votes
0 answers
21 views

Can I use a 1d convolution on a set of coordinates?

So i am training a reinforcement learning agent. It is fed in its position, and the target positions using x,y coordinates. An example input would be like [[1,3],[2,2],[5,1]]. I thought that since if ...
user avatar
0 votes
0 answers
8 views

multidimensional output from a DQN

The output of a DQN gives the Q value of each actions and it is an one dimensional vector. Can we get the output from a DQN as a matrix?
user avatar
  • 1
1 vote
0 answers
9 views

Custom Simulator for Deep Reinforcement Learning

I am trying to develop a control method for a specific process in industry. I have a time-series of data for the process and want to develop a prediction model base on attention mechanism to estimate ...
user avatar
0 votes
1 answer
23 views

Why DQN but no Deep Sarsa?

Why is DQN frequently used while there is hardly any occurrence of Deep Sarsa? I found this paper https://arxiv.org/pdf/1702.03118.pdf using it, but nothing else which might be relevant. I assume the ...
user avatar
  • 3
0 votes
0 answers
33 views

Unexpected keyword argument error in tensorflow-agents replay buffers

Following the tensorflow tutorial on deep reinforcement learning and DQN. Even after setting up the exact same libraries and running the same code, I am getting some error. ...
user avatar
  • 283
0 votes
0 answers
10 views

Multi-task reinforcement learning with different action spaces

I'm currently working on a project in which I need apply multi -task reinforcement learning. Over the same state space, each agent aims to do a separate task, but the action spaces of agents are ...
user avatar
  • 11
0 votes
0 answers
12 views

Mutli Armed Bandit : determine how to reward a selection

I am new to reinforcement learning and i am trying to understand more on how to apply multi armed bandit in real world cases. So here is my scenario, as i'm new on this i'm starting on small cases :- <...
user avatar
2 votes
1 answer
33 views

How to Form the Training Examples for Deep Q Network in Reinforcement Learning?

Trying to pick up basics of reinforcement learning by self-study from some blogs and texts. Forgive me if the question is too basic and different bits that I understand are a bit messy, but even after ...
user avatar
  • 283
0 votes
1 answer
21 views

Is it feasible to integrate convolutionnal layers as Reinforcement Learning input to learn video game?

Let's say, you want to apply reinforcement learning on a simple 2D game. (ex : super mario) The easy way is of course to retrieve an abstraction of the environnment, per example using gym and/or an ...
user avatar
0 votes
0 answers
10 views

Is smoothing required in reinforcement learning?

I am working on a chaotic time series of Cryptocurrencies and using reinforcement learning for allocating weights to different crypto assets. Is it incorrect if I use SG filter/exponential smoothing ...
user avatar
0 votes
0 answers
145 views

gym car racing v0 using DQN

I am currently learning reinforcement learning and wanted to use it on the car racing-v0 environment. I have successfully made it using PPO algorithm and now I want to use a DQN algorithm but when I ...
user avatar
  • 1
0 votes
0 answers
29 views

Reinforcement Learning vs Retraining

I have created a complex ML model using supervised learning. For the sake of discussion, let's say my model identifies dogs and a human labels the output as "correct" or "not correct&...
user avatar
0 votes
0 answers
19 views

TF Agents DdqnAgent for Continuous Tasks (Non-Episodic Environments)

I would like to use TF Agents in Non-Episodic environments (continuous tasks without a termination state). In such implementations, the agent can continue learning without the need to reset the ...
user avatar
0 votes
1 answer
13 views

experience replay memory: saving the next state required when state does not depend on action?

so, I am using an agent with a state-action-policy and I am trying to understand the concept of experience replay memory (ERM). As far as I learned until now, the ERM is basically a buffer that stores ...
user avatar
0 votes
0 answers
7 views

Is my exploration scheme in reinforcement learning done correctly?

so, I am training a deterministic policy, represented by basically a convolutional networks. I have an action space which is basically a vector of weights / probabilities, output by the network. The ...
user avatar
2 votes
0 answers
43 views

how to improve recall by retraining a model on its feedback

I am creating a supervised model using sensitive and scarce data. For the sake of discussion, I've simiplified the problem statement by assuming that I'm creating a model for identifying dogs. Let's ...
user avatar
0 votes
0 answers
58 views

Keras on-policy "Advantage Actor Critic" implementation

understand and implement on-policy "Advantage Actor-Critic" The Keras RL example is straight and simple, it uses The Kersa functional API to create an actor-critic and after each episode ...
user avatar
  • 1
1 vote
0 answers
28 views

Is it possible to use structured(tabular) data as a reinforcement learning environment?

I want to do an RL project in which the agent will learn to drop duplicates in a tabular data. But I couldn't find any examples of RL being used that way - checked the RL based recommendation systems ...
user avatar
  • 11
1 vote
0 answers
34 views

Why we use differential return with the average-reward settings for continuing tasks in reinforcement learning?

What is the intuition behind using differential return with the average-reward settings for continuing tasks in reinforcement learning? Why can't we use the return defined as before.
user avatar
  • 111
1 vote
0 answers
35 views

How to calculate the fully connected neural network footprint for each layer?

I'm using a MADDPG (RL algorithm) and trying to find out the memory footprint for each layer in it. Maddpg: https://github.com/openai/maddpg The neural network is described here. ...
user avatar
  • 11
1 vote
1 answer
29 views

How does bias and variance relate with the training/testing error in Machine Learning. In layman terms does high variance means high testing error

What causes high BIAS/VARIANCE and what are the consequences. Can some one explain in simple terms w.r.t to training/testing errors . Thanks
user avatar
0 votes
1 answer
45 views

Reinforcement Learning - Model based and model free

I'm studying reinforcement learning and I found confusing information. I know there are two different types of reinforcement learning, model based and model free. In the second image, it is possible ...
user avatar
  • 459
0 votes
0 answers
13 views

Chained Decisions in Reinforcement Learning

I am working on a project of portfolio optimization with reinforcement learning. I would like incorporate a dependent decision process: Decide which asset should be bought. Decide about the amount ...
user avatar
  • 3
0 votes
1 answer
19 views

Understanding derivation of gradient optimisation problem

I'm following a tutorial on youtube about reinforcement learning. They are going through the steps to understand policy gradient optimisation. In one of the steps he says ...
user avatar
  • 113
0 votes
0 answers
15 views

How to format inputs for recommender neural network

I am trying to figure out generally how a production scale recommender system could be designed around neural networks. In the case of a linear model, one could simply store the preferences weight ...
user avatar
0 votes
0 answers
16 views

Compute for policy the state value function v(s) for each state

The instruction of the question: State A is absorbing. Transition to A from state 1 or 4 yields an immediate reward of 12. All other transitions incur a reward of 1. Transitions are deterministic (i.e....
user avatar
0 votes
1 answer
50 views

How to get negative samples for reccomender system

In a recommendation system that is based on user preferences and item features (rather than a collaborative filtering approach), how might training be done if only positive samples can be found? For ...
user avatar
1 vote
0 answers
17 views

Given a set of options where one option is selected prior to an outcome, how to model optimal selection that will increase likelihood of (+) outcome

Say that we have a set of treatment plans (the options) available to a patient. Treatment plans can be invasive-surgery, no-surgery, less-invasive surgery ext... We have a dataset where a treatment ...
user avatar
0 votes
0 answers
8 views

Action-value estimation of deterministic policies with Monte Carlo method

In Monte Carlo-based action value estimation problem for a deterministic policy (estimation of $q_{\pi}(s,a)$),the estimation problem seems not to be well-defined because $q_{\pi}(s,a)$ by ...
user avatar
0 votes
1 answer
26 views

Deep RL: How often should retraining be done?

As the headline suggests, how often should retraining be performed when using deep RL? I guess retraining after every action is too expensive? I also guess there is no specific number (e.g. after 1,...
user avatar
  • 1
0 votes
0 answers
14 views

Stack as many industrial components as possible in a crate

The exact problem is a crate of industrial parts, made by injection molding in very high quantities. The objective is to put as much parts as possible in one crate. This is done by a small robotic arm ...
user avatar
  • 111
0 votes
0 answers
11 views

Value function when the policy is deterministic

This is the value function expression for a stochastic policy: $\displaystyle v_{\pi}(s)=\sum_{a \in \mathcal{A}}\pi(a|s)\bigg(\mathcal{R}_s^a+\gamma \sum_{s' \in \mathcal{S}} \mathbb{P}_{ss'}^a v_{\...
user avatar
1 vote
0 answers
23 views

Jacks car rental problem: why deterministic policies?

In Sutton & Barto Book: Reinforcement Learning: An Introduction, there is the following problem: I have this question: why are the policies to be considered here are deterministic?
user avatar
1 vote
0 answers
19 views

Which ML to use for search suggestion?

Problem: I want to create a program to organize text information and fast access to relevant documents. I would like to train a ML model to analyse the current situation and to suggest the next ...
user avatar
0 votes
0 answers
9 views

How to add constraints/restrictions to policy based reinforcement learning?

So we are trying to create an actor-critic policy reinforcement learning algorithm in which a portfolio of assets has to be selected. We would like to add the restriction that certain assets in that ...
user avatar
3 votes
2 answers
1k views

Can Reinforcement Learning learn to be deceptive?

I have seen several exampled of deploying RL agents in deceptive environnement or games and the agent learns to perform its tasks regardless. What about the other way around? Can RL be used to create ...
user avatar
  • 47
4 votes
3 answers
132 views

Which ML approach is the best for huge state spaces?

My issue derives from the challenge of solving a seemingly easy-looking game. To spare you the full catalogue of rules, here is a short summary of the game: Single player card game You go through a ...
user avatar
0 votes
1 answer
14 views

Output representation for a neural network to learn grid-based game with multiple units

I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
user avatar
  • 101
0 votes
0 answers
10 views

Computing the state-value function of a Markov decision process from the classical definition

For the above Markov decision process under given action policy $a_1$, how can I determine the value of state $s_1$ using the state-value definition $v(s)=E[G_t| S_t=s]$ where $G_t$ is the return? ...
user avatar
0 votes
1 answer
40 views

how to do feature engineering on Atari Pong game in code?

In RL context, I know that features are explanatory variables that represent or describe the states. If I want to do feature engineering on atari games and use it to solve RL task, how should I ...
user avatar
  • 101

1
2 3 4 5
12