Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0
votes
0answers
14 views

Deep RL: How often should retraining be done?

As the headline suggests, how often should retraining be performed when using deep RL? I guess retraining after every action is too expensive? I also guess there is no specific number (e.g. after 1,...
0
votes
0answers
10 views

Stack as many industrial components as possible in a crate

The exact problem is a crate of industrial parts, made by injection molding in very high quantities. The objective is to put as much parts as possible in one crate. This is done by a small robotic arm ...
0
votes
0answers
8 views

Value function when the policy is deterministic

This is the value function expression for a stochastic policy: $\displaystyle v_{\pi}(s)=\sum_{a \in \mathcal{A}}\pi(a|s)\bigg(\mathcal{R}_s^a+\gamma \sum_{s' \in \mathcal{S}} \mathbb{P}_{ss'}^a v_{\...
1
vote
0answers
18 views

Jacks car rental problem: why deterministic policies?

In Sutton & Barto Book: Reinforcement Learning: An Introduction, there is the following problem: I have this question: why are the policies to be considered here are deterministic?
1
vote
0answers
16 views

Which ML to use for search suggestion?

Problem: I want to create a program to organize text information and fast access to relevant documents. I would like to train a ML model to analyse the current situation and to suggest the next ...
0
votes
0answers
5 views

How to add constraints/restrictions to policy based reinforcement learning?

So we are trying to create an actor-critic policy reinforcement learning algorithm in which a portfolio of assets has to be selected. We would like to add the restriction that certain assets in that ...
3
votes
2answers
953 views

Can Reinforcement Learning learn to be deceptive?

I have seen several exampled of deploying RL agents in deceptive environnement or games and the agent learns to perform its tasks regardless. What about the other way around? Can RL be used to create ...
4
votes
3answers
111 views
+100

Which ML approach is the best for huge state spaces?

My issue derives from the challenge of solving a seemingly easy-looking game. To spare you the full catalogue of rules, here is a short summary of the game: Single player card game You go through a ...
0
votes
0answers
6 views

Output representation for a neural network to learn grid-based game with multiple units

I have a round based game played on a grid map with multiple units that I would like to control in some fashion using neural network (NN). All of the units are moved at once. Each unit can move in any ...
0
votes
0answers
6 views

Computing the state-value function of a Markov decision process from the classical definition

For the above Markov decision process under given action policy $a_1$, how can I determine the value of state $s_1$ using the state-value definition $v(s)=E[G_t| S_t=s]$ where $G_t$ is the return? ...
0
votes
1answer
15 views

how to do feature engineering on Atari Pong game in code?

In RL context, I know that features are explanatory variables that represent or describe the states. If I want to do feature engineering on atari games and use it to solve RL task, how should I ...
1
vote
0answers
14 views

Reinforcement learning in real world without OpenAI Gym

I have been searching for reinforcement learning libraries or examples that aren't using a simulated environment like OpenAI Gym. I havn't been successful at all until yet and I would really ...
0
votes
1answer
23 views

Why Do We Store The action In Replay Memory Deep Q-learning

According to my understanding, in Deep Q-learning, in order to train the NN, the agent stores experiences in a buffer (Replayed Memory), and each experience contains: ...
0
votes
0answers
17 views

How contextual bandit use single model out put q-value for each arm?

I've been studying Thompson Sampling and one of the bottleneck is that when recommending 100 items to users, each item has beta distribution thus need to select ...
0
votes
0answers
6 views

Q-learning episode and relation to convergence in MY scenario?

I used Q-learning for routing. I have used the Bellman equation. I have certain other technical aspects in the code that add some novelty. But I have mixed doubts regarding episode and corresponding ...
1
vote
0answers
12 views

How to understand expected value function in a stochastic policy setting?

I am currently reading "Deep Reinforcement Learning with Python" by Sudharsan Ravichandiran. I am still on the first chapter of Introduction and understanding the most basic concepts. He has ...
1
vote
1answer
20 views

Difference between regret and pseudo-regret definitions in multi-armed bandits

I posted this question Cross Validated, but didn't get any answer. So I am posting it here too, as the question is very relevant to machine learning I am following the book Bandit Algorithms. In page ...
0
votes
0answers
35 views

Reinforcement learning policy gradient derivation

I was reading a document about Reinforcement Learning policy gradient http://web.stanford.edu/class/cs234/CS234Win2019/slides/lnotes8.pdf when I encountered this expression $ \nabla_{\theta} \mathbb{...
0
votes
0answers
68 views

Error:DQN expects a model that has one dimension for each action, in this case (1, 2, 1, 0)

I am building an RL agent for which the model is defined: ...
0
votes
0answers
21 views

Convergence of Sarsa($\lambda$)

Is there any theorem on the convergence of the Sarsa($\lambda$) Algorithm? I am currently working through the theory of Reinforcement Learning with the lecture by David Silver and the book of Sutton &...
0
votes
2answers
61 views

Is reinforcement learning analogous to stochastic gradient descent?

Not in a strict mathematical formulation sense but, would there be there any key overlapping principals for the two optimisation approaches? For example, how does $$\{x_i, y_i, \mathrm{grad}_i \}$$ (...
0
votes
0answers
16 views

How to test/use trained reinforcement learning model?

I have now gone trough lots of examples and tutorials but they all end up with plotting the learning process or/and a video that I am not totally sure what it is representing. Example: https://www....
1
vote
0answers
74 views

Soft actor-critic reinforcement learning for 100x100 maze environment

I am doing a project which requires a soft actor-critic reinforcement learning agent to learn how to reach a goal in a 100x100 maze environment as the one below: The state space is discrete and only ...
1
vote
0answers
15 views

Neural network predicting on similar inputs with many features

I am training a deep RL agent (DQN) on states with 333 components that usually are very similar between themselves. The actions predicted by the agent, which is nothing but the max operator applied to ...
0
votes
2answers
68 views

Reinforcement learning algorithms

I want to study and develop some application using reinforcement learning machine learning methods. I am already familiar with classification problem using supervised learning. Can someone suggest me ...
3
votes
0answers
28 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
1
vote
1answer
34 views

How to solve a non classification problem with multiple plausible reults?(Tensorflow)

I'm fairly new to ML and now that I digged through tutorials and documentations I wanted to create a model myself now. The problem: I am a carpenter and back then in shool we had a problem where we ...
0
votes
0answers
9 views

RL agent behave differently for different data

I am training an RL model using PPO for AAPL stock. There are 3 actions to take, Buy, Sell or Hold. If there is a Buy(/Sell) signal, the environment will buy(/sell) all. To trade for each year, the ...
1
vote
1answer
29 views

How to install custom pakages on Google-Colaboratory

I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'. I done the following things. First I cloned the git hub repository ...
0
votes
0answers
17 views

Improving the Actor Critic algorithm proposed by Keras

In this page of keras's website, a reinforcement learning algorithm based in an actor critic scheme has been described. It is a deep policy gradient algorithm (hence DPG). Of course keras functions ...
0
votes
0answers
18 views

build Environment using OpenAI Gym

I have an environment which contains 26 states. Each episode sample have an terminal state.I want my agent to learn how to get to the terminal state faster by using the minimum sequences of action. ...
0
votes
0answers
9 views

How to calculate relation function of State and Action in delayed action effect Environment?

I'm trying to calculate corr-coef(or other good relation function) of State and Action in 'delayed action effect' Envrironment. In this environment, agent observes states, then it returns action and ...
3
votes
1answer
72 views

Machine learning roadmap not for beginners [closed]

To introduce myself: I know what is RL, know some RL algorithms such as PPO, A2C. Know about offline RL, online RL. I have read many papers about RL. Such as MuZero, AplhaZero, Decision Transformer ...
1
vote
1answer
15 views

Do the examples of reinforcement learning such as the hide and seek learn to solve anything but the environment they start in?

As pictured here. https://www.youtube.com/watch?v=kopoLzvh5jY OpenAI implements a study for reinforcement learning with multiple learning agents which appear adversarial. "Millions of rounds&...
2
votes
0answers
12 views

Building a simulator for continuous state, discrete action reinforcement learning

I am trying to build a simulator that optimizes the performance and temperature of a device. I want the device to perform well, but without making the device too hot. If the device becomes too hot, I ...
0
votes
1answer
44 views

How to write a reward function that optimizes for profit and revenue?

So I want to write a reward function for a reinforcement learning model which picks products to display to a customer. Each product has a profit margin %. Higher price products will have a higher ...
0
votes
0answers
33 views

How to train a policy and a value network, implementing alphazero at chess

So, I'm trying to implement alphazero's logic on the game of chess. What I understand so far of the algorithm is: Load 2 models, one of which is the best model you have so far. Both these models have ...
0
votes
0answers
10 views

Video games with built-in logging

I'd like to get some practice with reinforcement learning and data analysis in the domain of video games. I'm looking for a game that logs user behaviours (e.g. UI click events, player position, item ...
1
vote
1answer
41 views

Does GPU decreases training time for on-policy RL?

I was wondering whether using a GPU will be effective if I am using an on-policy (eg PPO) RL as the model? I.e, how can we use a GPU to decrease training time for an on-policy RL model? I recently ...
1
vote
1answer
47 views

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use ...
1
vote
1answer
23 views

Is it possible to optimize the Client Lifetime Value with Reinforcement Learning using Marketing Activities as Actions?

I have been researching the Reinforcement Learning topic. I have been looking if this is the correct way to optimize the marketing actions of my company given that we are looking to optimize the ...
1
vote
1answer
45 views

What approach should I take for my product classification ML model with user feedback for improving result accuracy?

I'm trying to implement a product categorization ML model on a dataset with the following structure: Data sample I want to my model to be able to predict the correct category that the product should ...
1
vote
0answers
14 views

How to create transition probability (state) for q-learning algorithm designed to control traffic light system using python?

I am trying to create a q learning algorithm to control traffic light systems. I am representing the state with a matrix : ...
0
votes
0answers
22 views

how to in enhance A3C entropy?

I'm trying to implement this A3C code in my custom environment, and I have a basic understanding of the algorithm. The algorithm worked, but it did not give me a good performance. I looked into ...
0
votes
0answers
23 views

How to draw Reward vs Episode Graph

I have just started learning reinforcement learning and I was following some tutorials on youtube. I found out out that no one was explaining how to draw reward vs episode graph. So I concluded that ...
0
votes
0answers
27 views

Which type of Reinforcement Learning to use?

First of all, I am new to the topic of machine learning, AI, etc. However, I would like to create a sophisticated decision algorithm. The decision is a Boolean decision and you can suppose the result ...
0
votes
0answers
52 views

Why is Pytorch 100x faster than Tensorflow for Deep Reinforcement Implementation?

[Intro] I am currently getting into machine learning coming from mathematics and am following a few courses to get the hang of this. In one of the courses, the author has a github with code examples: ...
-1
votes
1answer
44 views

What problem can solve with reinforcement learning

I think classification problem cannot solve with reinforcement learning. I don't know what problem can solve with reinforcement learning. Is there a way to see if this problem can be solved with ...
0
votes
0answers
16 views

How to solve Bellman-Hamiltonian-Jacobi in stochastic environment through machine learning?

Suppose there is a stochastic optimal control problem where the uncertainty is because of the random arrival of entities into our system which is modelled as a Poisson process with rate $\lambda(p)$. ...
0
votes
0answers
15 views

Reinforcement learning with multiple agents

I am starting a project which involves multiple agents, some that play on the same team and others that play on the opponent's team. Since I am rather new to this topic, I did some research to get an ...

1
2 3 4 5
12