Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0
votes
0answers
11 views

Reinforcement learning policy gradient derivation

I was reading a document about Reinforcement Learning policy gradient http://web.stanford.edu/class/cs234/CS234Win2019/slides/lnotes8.pdf when I encountered this expression $ \nabla_{\theta} \mathbb{...
0
votes
0answers
13 views
0
votes
0answers
16 views

Convergence of Sarsa($\lambda$)

Is there any theorem on the convergence of the Sarsa($\lambda$) Algorithm? I am currently working through the theory of Reinforcement Learning with the lecture by David Silver and the book of Sutton &...
0
votes
2answers
44 views

Is reinforcement learning analogous to stochastic gradient descent?

Not in a strict mathematical formulation sense but, would there be there any key overlapping principals for the two optimisation approaches? For example, how does $$\{x_i, y_i, \mathrm{grad}_i \}$$ (...
0
votes
0answers
16 views

How to test/use trained reinforcement learning model?

I have now gone trough lots of examples and tutorials but they all end up with plotting the learning process or/and a video that I am not totally sure what it is representing. Example: https://www....
1
vote
0answers
66 views

Soft actor-critic reinforcement learning for 100x100 maze environment

I am doing a project which requires a soft actor-critic reinforcement learning agent to learn how to reach a goal in a 100x100 maze environment as the one below: The state space is discrete and only ...
1
vote
0answers
15 views

Neural network predicting on similar inputs with many features

I am training a deep RL agent (DQN) on states with 333 components that usually are very similar between themselves. The actions predicted by the agent, which is nothing but the max operator applied to ...
1
vote
2answers
57 views

Reinforcement learning algorithms

I want to study and develop some application using reinforcement learning machine learning methods. I am already familiar with classification problem using supervised learning. Can someone suggest me ...
3
votes
0answers
24 views

Reinforcement Learning applied to Optimisation Problem

Problem Statement: We are given an optimisation problem; with production centres, source airport, destination airports, transfer points and finally delivered to the customers. This is better explained ...
1
vote
1answer
33 views

How to solve a non classification problem with multiple plausible reults?(Tensorflow)

I'm fairly new to ML and now that I digged through tutorials and documentations I wanted to create a model myself now. The problem: I am a carpenter and back then in shool we had a problem where we ...
0
votes
0answers
7 views

RL agent behave differently for different data

I am training an RL model using PPO for AAPL stock. There are 3 actions to take, Buy, Sell or Hold. If there is a Buy(/Sell) signal, the environment will buy(/sell) all. To trade for each year, the ...
1
vote
1answer
23 views

How to install custom pakages on Google-Colaboratory

I'm trying to install packages on GOOGLE COLAB, but I'm facing Import error, I can't import Sub module of my main module 'gym'. I done the following things. First I cloned the git hub repository ...
0
votes
0answers
11 views

Improving the Actor Critic algorithm proposed by Keras

In this page of keras's website, a reinforcement learning algorithm based in an actor critic scheme has been described. It is a deep policy gradient algorithm (hence DPG). Of course keras functions ...
0
votes
0answers
17 views

build Environment using OpenAI Gym

I have an environment which contains 26 states. Each episode sample have an terminal state.I want my agent to learn how to get to the terminal state faster by using the minimum sequences of action. ...
0
votes
0answers
8 views

How to calculate relation function of State and Action in delayed action effect Environment?

I'm trying to calculate corr-coef(or other good relation function) of State and Action in 'delayed action effect' Envrironment. In this environment, agent observes states, then it returns action and ...
3
votes
1answer
64 views

Machine learning roadmap not for beginners [closed]

To introduce myself: I know what is RL, know some RL algorithms such as PPO, A2C. Know about offline RL, online RL. I have read many papers about RL. Such as MuZero, AplhaZero, Decision Transformer ...
1
vote
1answer
15 views

Do the examples of reinforcement learning such as the hide and seek learn to solve anything but the environment they start in?

As pictured here. https://www.youtube.com/watch?v=kopoLzvh5jY OpenAI implements a study for reinforcement learning with multiple learning agents which appear adversarial. "Millions of rounds&...
2
votes
0answers
12 views

Building a simulator for continuous state, discrete action reinforcement learning

I am trying to build a simulator that optimizes the performance and temperature of a device. I want the device to perform well, but without making the device too hot. If the device becomes too hot, I ...
0
votes
1answer
41 views

How to write a reward function that optimizes for profit and revenue?

So I want to write a reward function for a reinforcement learning model which picks products to display to a customer. Each product has a profit margin %. Higher price products will have a higher ...
0
votes
0answers
29 views

How to train a policy and a value network, implementing alphazero at chess

So, I'm trying to implement alphazero's logic on the game of chess. What I understand so far of the algorithm is: Load 2 models, one of which is the best model you have so far. Both these models have ...
0
votes
0answers
9 views

Video games with built-in logging

I'd like to get some practice with reinforcement learning and data analysis in the domain of video games. I'm looking for a game that logs user behaviours (e.g. UI click events, player position, item ...
2
votes
1answer
33 views

Does GPU decreases training time for on-policy RL?

I was wondering whether using a GPU will be effective if I am using an on-policy (eg PPO) RL as the model? I.e, how can we use a GPU to decrease training time for an on-policy RL model? I recently ...
2
votes
1answer
44 views

Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use ...
1
vote
1answer
21 views

Is it possible to optimize the Client Lifetime Value with Reinforcement Learning using Marketing Activities as Actions?

I have been researching the Reinforcement Learning topic. I have been looking if this is the correct way to optimize the marketing actions of my company given that we are looking to optimize the ...
1
vote
1answer
36 views

What approach should I take for my product classification ML model with user feedback for improving result accuracy?

I'm trying to implement a product categorization ML model on a dataset with the following structure: Data sample I want to my model to be able to predict the correct category that the product should ...
1
vote
0answers
14 views

How to create transition probability (state) for q-learning algorithm designed to control traffic light system using python?

I am trying to create a q learning algorithm to control traffic light systems. I am representing the state with a matrix : ...
0
votes
0answers
21 views

how to in enhance A3C entropy?

I'm trying to implement this A3C code in my custom environment, and I have a basic understanding of the algorithm. The algorithm worked, but it did not give me a good performance. I looked into ...
0
votes
0answers
12 views

How to draw Reward vs Episode Graph

I have just started learning reinforcement learning and I was following some tutorials on youtube. I found out out that no one was explaining how to draw reward vs episode graph. So I concluded that ...
0
votes
0answers
27 views

Which type of Reinforcement Learning to use?

First of all, I am new to the topic of machine learning, AI, etc. However, I would like to create a sophisticated decision algorithm. The decision is a Boolean decision and you can suppose the result ...
0
votes
0answers
43 views

Why is Pytorch 100x faster than Tensorflow for Deep Reinforcement Implementation?

[Intro] I am currently getting into machine learning coming from mathematics and am following a few courses to get the hang of this. In one of the courses, the author has a github with code examples: ...
-1
votes
1answer
30 views

What problem can solve with reinforcement learning

I think classification problem cannot solve with reinforcement learning. I don't know what problem can solve with reinforcement learning. Is there a way to see if this problem can be solved with ...
1
vote
0answers
16 views

How to solve Bellman-Hamiltonian-Jacobi in stochastic environment through machine learning?

Suppose there is a stochastic optimal control problem where the uncertainty is because of the random arrival of entities into our system which is modelled as a Poisson process with rate $\lambda(p)$. ...
1
vote
0answers
15 views

Reinforcement learning with multiple agents

I am starting a project which involves multiple agents, some that play on the same team and others that play on the opponent's team. Since I am rather new to this topic, I did some research to get an ...
2
votes
0answers
28 views

Multi-arm bandits

I would like to model a problem as a multi-armed bandit problem. In the data we have contextual information (user demographics, preferences, etc.) but this contextual information of each user is not ...
0
votes
0answers
26 views

Reinforcement Learning, wont learn and bad in test set

I'm study and try to understand better the reinforcement learning branch; In this case I want to learn the agent to make a reward; I've tried with: A2C DQN PPO2 but the agent in test env make ever ...
0
votes
1answer
41 views

Deep Q-learning

I am working on the DDQN algorithm which is given in the following paper. I am facing a problem with the Q value. The author calculate Q value by this Q(s, a; θ , α, β) = V(s; θ , β) + A(s, a; θ , α). ...
0
votes
0answers
31 views

Can one use reinforcement learning to find a fixed point of a dynamical system?

I would like to know if it is possible to use reinforcement learning (RL) to find a fixed point of a dynamical system. By the fixed point, I mean $f(x)=x$. One can use Newton's method to get the fixed ...
0
votes
0answers
35 views

Q value is estimated under state V value and action A value for DDQN

How Q value is estimated under state V value and action A value. Given the below DDQN algorithm, the deep network is divided into two parts on the end layer, including state value function V(s) which ...
1
vote
1answer
44 views

Reinforcement Learning - after 800k timesteps agent wont learng

I'm study and try a reinforcement learning. now im using gym and stable-baseline. My project have a step where I calculate a reward with a function. -> step() -> calculate_reward() -> return ...
0
votes
0answers
8 views

Action signal gets saturated too quickly at DDPG

I am new to RL, so there might be some things that I miss here. My basic setup is like this: I have 60 observations and 15 actions, and I am trying to train a very nonlinear system with an DDPG agent, ...
0
votes
0answers
23 views

external input and Reinforcement Learning

Is there an RL method which in it the next state depends on the "current action" and "current state" AND an "External Input"?
0
votes
1answer
34 views

How does using another agents action choice impact the efficacy of learning with Deep Reinforcement Learning

I am doing a project where I have multiple soft actor-critic sub-agents learning at the same time in an environment using shared experiences. Each sub-agent selects an action using their own policy, ...
0
votes
1answer
36 views

Understanding DQN Algorithm

Im studying the deep q learning algorithm. You can see it in the picture here: DQN I have a few questions about the deep q learning algorithm. What do they mean with row 14: ...
0
votes
0answers
9 views

reinforcement learning , python, problem empty list

I'm trying to simulate a model, where a vehicle after having been in one destination then goes to the other on the list. The problem is that the list is empty. Someone help me? ...
0
votes
0answers
21 views

Where can I find real world examples of feature construction for reinforcement learning?

Since I am relatively new to the field of reinforcement learning and I read through the classic book Reinforcement Learning: An Introduction by Sutton et. al., in Chapter 9 the authors mention several ...
0
votes
0answers
6 views

Reinforcement Learning for distributing items on multiple places, i.e., scale balancing

I am interested in creating a reinforcement learning algorithm for assigning items to different buckets, such that the buckets are almost the same weight, i.e., scale but with more than two places to ...
0
votes
0answers
12 views

Usage of Chainer for Reinforcement Learning Purposes

I am trying toreplicate this code https://www.kaggle.com/prithviraj7387/deep-reinforcement-learning-on-stock-data used for Reinforcement Learning which is used for buying/selling stock I wanted to ...
1
vote
1answer
23 views

Is reinforcement learning suitable for the Dial-a-Ride problem?

Is reinforcement learning suitable for this problem or will it perform poorly against classical algorithms? "The Dial-a-Ride Problem (DARP) consists of designing vehicle routes and schedules for ...
0
votes
0answers
9 views

Algortihm for distributing volume in 1min stock intervals

Context: I have historical 1min prices for stocks, including premarket. However, when importing real-time data, the standard practice in the financial data industry is to give only OHLC (open, high, ...
0
votes
0answers
31 views

Am I using this neural network in a wrong way?

I'm trying to solve a RL problem; the Contextual Bandit problem using Deep Q Learning. My data is all simulated. I have this environment: ...

1
2 3 4 5
12