Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [reinforcement-learning]

Area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

"Unique" or "Repeated" experiences in memory replay...?

I'm training an RL agent/model (DRL/DQN). Say that, for each learning step, the memory replay used by the agent to learn, has N elements (experiences) stored, where only X are unique elements (...
Jose Alberto Salazar's user avatar
0 votes
0 answers
12 views

Reinforcement learning give reward after finishing the data classification instead of acting one by one and CNN based reinforcement learning

I am trying to write an reinforcement based based trading system and while trying to do that, the only way that I can do and I found is rewarding the model for each action but it actually performs bad ...
ugroon's user avatar
  • 1
0 votes
0 answers
23 views

Using Reinforcement learning for minimisation

I would like to use reinforcement learning for the optimisation of a given function under some contraints. Take for example the following problems: ...
Ach Raf's user avatar
0 votes
0 answers
26 views

Combining monte carlo with deep learning to improve the estimation

I am in situation where i need to estimate the attenuation of an EM wave . we consider EM wave as collection of photons. These photons when strike with some dust particles they scatter in different ...
user7341333's user avatar
1 vote
1 answer
43 views

In reinforcement learning, why learn Q rather than V?

Why do we learn the action-value function $Q(s,a)$ rather than the (just state) value function $V(s)$? At least for deterministic environments? $V$ is much smaller than $Q$, and they are trivially ...
xyzzyrz's user avatar
  • 113
1 vote
0 answers
24 views

Pretrain a Stable Baseline 3 SAC algo from recorded historical / real-life data

I currently have a database recorded from human behavioural (observation, action, reward, next action) with 400k examples collected in real-life conditions. I want to implement a kind of pretraining ...
GerardL's user avatar
  • 13
0 votes
0 answers
42 views

Getting actions probabilities instead of an unique prediction in Stable Baselines 3 SAC?

I try to understand how getting an actions probability table instead of an unique prediction in stable baselines 3 SAC in order to override 'predict' method to filter invalid actions. I guess the good ...
GerardL's user avatar
  • 13
0 votes
0 answers
46 views

Feature extractor for a a 2D array input

I want to train a SAC algorithm from Stable Baselines 3 with a (100,210) shaped array as input. The array is a stack of observations cumulated along axis 0. The last row is current observation. SAC ...
GerardL's user avatar
  • 13
0 votes
1 answer
64 views

Illegal action reward strategy for reinforcement learning : reward shaping and termination / truncation

I have some questions about strategy to adopt regarding illagal action handling in reinforcement learning (Stable Baselines 3 / SAC algo). First is about reward shaping, second is about terminating / ...
GerardL's user avatar
  • 13
0 votes
1 answer
27 views

Learning the gradient descent stepsize with RL [closed]

Problem statement: I've been working on a project to accelerate the convergence of gradient descent using reinforcement learning (RL). I want to learn a policy that can map the current state of ...
CodeGuy's user avatar
0 votes
0 answers
49 views

RLHF fine-tune llama2 in vertex ai

I have fine tune RLHF with Vertex AI Pipeline. But deployed model not showing in model registry. Why? code i have used: ...
Sandun Tharaka's user avatar
1 vote
1 answer
31 views

Why don't search engines filter out unethical/illegal searches?

(Not sure if this question is appropriate to this SE) I'm studying the LLMs course on Coursera. One topic they deal with is how to get the LLM to not respond with unethical/illegal things, e.g. if you ...
Allure's user avatar
  • 275
1 vote
0 answers
34 views

Building an engine for a chess-like game

I'm working on building an AI for a chess-like game. I've implemented a Monty Carlo Tree Search (for the early game) and Rainbow DQN (for the mid to late game), and will be implementing Alpha Beta ...
william paine's user avatar
0 votes
0 answers
11 views

Passing the Parallel API tests in PettingZoo for custom multi-agent environment

from pettingzoo.test import ( parallel_api_test, parallel_seed_test, max_cycles_test, performance_benchmark, ) I have a custom multiagent ...
hridayns's user avatar
  • 121
0 votes
0 answers
9 views

DQN with action (obtained from some baseline algorithm) as part of input

I have some base algorithm (that chooses an action and is static but based on rigorous statistical theory). This action is between 0 and 1 and continous, $$ a_{baseline} \in (0,1) .$$ This means that ...
cactus_splines's user avatar
0 votes
1 answer
40 views

Can I compare Q-VALUE for the same action across different states?

Dears, I'm new with RL and try to apply to my project. I've run RL with some example data and got the My question is if I could compare the Q-values for the same current action across different prior ...
tree's user avatar
  • 1
0 votes
0 answers
133 views

Runtime Error: one of the variables needed for gradient computation has been modified by an inplace operation:

I have the following code for a reinforcement learning using proximal policy optimization. It gives the following run time error. ...
heyula's user avatar
  • 37
2 votes
1 answer
61 views

How to represent output layer if the action size changes dynamically?

I am new to ML and I need to train a chess agent using proximal policy optimization. Board is represented as string and the environment gives a list of valid moves for each step. ...
heyula's user avatar
  • 37
0 votes
1 answer
28 views

Value Iteration

For value iteration problem, using Bellman equation for iteration 2 , it is clear to me how values are getting calculated. But for iteration 3 values are unclear to me. Can one help me in this
Bosch Reserach's user avatar
0 votes
0 answers
61 views

Examples of decision processes where action has no effect on the next state, but has on the reward?

Within my studies, I work on a recurrent reinforcement learning project and I struggle to find real-world problems with a property that is important to my solutions. I look for instances of problems ...
Tomasz Witkowski's user avatar
0 votes
0 answers
16 views

Reinforcement Learning: Formally, does an exponentially decaying epsilon satisfy GLIE?

I am aware that exponentially decaying exploration constants (epsilon) are used practically. Formally, though, do they satisfy the GLIE condition? Specifically, relating this to the Borel-Cantelli ...
jeremy.ebg's user avatar
0 votes
0 answers
59 views

Should offline algorithms (SAC, DDGP, TD3) be used with suprocesses vectorized environements?

I read most of offline algorithms from Stable Baselines 3 should not be subprocessed (see here : https://stable-baselines3.readthedocs.io/en/master/guide/rl_tips.html - "Which algorithm should I ...
GerardL's user avatar
  • 13
0 votes
0 answers
44 views

What are typical hyperparameter ranges and significance for SB3's offline algorithm?

I'm trying to perform an hyperparameter tuning on a SAC algorithm (Stable Baselines 3) with optuna lib. I guess hyperparameters have different significance and typical ranges depending of the selected ...
GerardL's user avatar
  • 13
0 votes
0 answers
38 views

Machine learning can be broken down into supervised, unsupervised, and reinforcement learning. Is there anything else?

Is it even logically possible to have a type of machine learning that doesn't fall into those three paradigms? Supervised: a dataset of inputs and outputs are fed to an algorithm which learns a ...
Austin Capobianco's user avatar
0 votes
1 answer
41 views

Reinforcement learning

I am working on a sentiment analysis project. I used BERT model for training but lack of data it gives huge overfitting. So after i moved LLM approach to do that. Using LLM finally i got good results....
Sandun Tharaka's user avatar
0 votes
0 answers
34 views

Bootstrapping formula for TD learning in Reinforcement Learning

In Reinforcement Learning (Sutton & Barto, 2018), p.120, equations (6.3)-(6.4) , to explain the idea of bootstrapping in Temporal-difference learning: \begin{equation} v_{\pi}(s) := E_{\pi}[G_t|...
fermented_bean's user avatar
0 votes
1 answer
63 views

Deep Q-Learning: How are network parameters updated, and why consider episodes in the first place?

I'm trying to wrap my head around the implementation of deep $Q$-learning, and why we even consider episodes in the first place. The usual set-up is that we initialize some starting state $s_0$, then ...
infinitylord's user avatar
0 votes
0 answers
48 views

Is n-Step SARSA Off-Policy?

To keep things simple, let us consider the case of an episodic Finite MDP with n=2 and no discounts. Copying the equation for n-step SARSA from Reinforcement Learning 2nd edition, page 146: The n-step ...
mashrivas's user avatar
0 votes
0 answers
29 views

Building an application to group/label browser tabs

I'm trying to develop an application where users can click a button and all of the open tabs in their browser will be placed into tab groupings based on similarity of the tab. Microsoft Edge has a ...
Ben's user avatar
  • 1
0 votes
0 answers
21 views

SARSA with Updated Actions

I am trying to gain a better understanding of the Classic SARSA Algorithm. Let us assume Finite MDP which is simple enough so that we can use Tabular methods, so no function approximation. In the ...
mashrivas's user avatar
0 votes
0 answers
22 views

Query modification for elastic search using machine learning

I have a problem statement that I'm struggling to formulate as a machine learning framework. There is a huge client database of documents - we're trying to come up with an efficient way of querying ...
user9343456's user avatar
1 vote
1 answer
153 views

Given historical states and action/reward data, is reinforcement learning a reasonable approach?

To summarize my problem: I want to maximize my total reward over all timesteps I have 3 discrete actions at each time step. The state vector for each time step has 5 features. The features are ...
dcl's user avatar
  • 251
0 votes
1 answer
339 views

Workflow for reinforcement learning in Python

I'm new to reinforcement learning but (I believe) I have certain experience with machine learning. I know if you work on, say, a classification problem, your workflow will be (in a very high level) ...
ElonMuskofBadIdeas's user avatar
0 votes
0 answers
683 views

Choosing the number of episodes and iterations when training a RL model

I know the parameters chosen for training a RL model depend heavily on the model itself as well as the problem. Nevertheless, I am trying to train a bunch of these agents on different environments, ...
user152104's user avatar
0 votes
0 answers
23 views

Can somebody explain to me, how the Hyperparameters in this expamle work? I only uderstood the meaning for gamma

I am quite new to this topic, but I want to understand how Reinforcement Learning (RL) works in this example (https://gymnasium.farama.org/tutorials/training_agents/reinforce_invpend_gym_v26/). I have ...
Peter's user avatar
  • 1
0 votes
1 answer
45 views

Stick to R or switch to Python as a newby in Reinforcement Learning?

Background I would consider myself to have a decent level in R and virtually 0 skills in any other programming language - something quite common in my field. Note also that my background is not in ...
ionatura's user avatar
0 votes
0 answers
233 views

Reinforcement learning with PPO - tips and tricks

I am trying to use PPO where the agent has to maneuver around an obstacle towards the target while respecting the spatial boundaries. While the agent learns to respect spatial boundaries it never ...
SathukaBootham's user avatar
2 votes
0 answers
20 views

Consistency error in visualization of policy improvement in Sutton & Barto's book?

Sutton & Barto introduce in their foundational book on "Reinforcement Learning: An Introduction" in the context of Dynamic Programming algorithms for policy evaluation and improvement. ...
Steve's user avatar
  • 41
0 votes
0 answers
29 views

Pretrained model for RNN Encoder-Decoder?

Our team are implementing a paper called Cold-Start-Reinforcement-Learning-with-Softmax-Policy-Gradient. Although the paper didn't mention. We want to use a pre-trained model, which is a RNN Encoder-...
jackson's user avatar
  • 25
0 votes
1 answer
53 views

Gradients Exploding with Custom Gradient and 2 layer MLP

I am training a 2 layer MLP on an off-policy learning-to-rank task, where the input is a list of documents against a query with a feature vector for each query-document pair, i.e. if there are M ...
SHASHANK GUPTA's user avatar
0 votes
1 answer
43 views

Inverse reinforcement learning with trajectories only

Inverse reinforcement learning (IRL) is a task that can learn a reward from other agent behaviour. Most IRL paradigms assume that dynamics of environment is known, that is the transition probability ...
JunjieChen's user avatar
0 votes
0 answers
26 views

Is it true that a multi-armed-bandit selects an action based on the future reward?

During my machine learning lecture I saw the following statement on the slides: "Multi-armed-bandit selects an action based on the future reward" Is that statement true? In my opinion it is ...
user150593's user avatar
0 votes
0 answers
33 views

Dueling DQN with varying number of actions

I have an RL problem, where the number of actions depends on the state. Furthermore, each action-value computation requires action information in the form of a high-dimensional, continuous vector in ...
WolfSovereign's user avatar
1 vote
0 answers
69 views

A working tf_agents.agents.PPOClipAgent example for reference?

I am really struggling on finding a working example for tf_agents.agents.PPOClipAgent, for instance for the simple CartPoleV1 environment. Does anybody have one or a link to a good example? I tried to ...
fede72bari's user avatar
0 votes
1 answer
423 views

Compatibility of anytrading Gym environment with TF-Agents

All Gym/Gymnasium standard environments are compatible with TwnsorFlow RL agents, but when I tried to use TF-Agents with anytrading I get errors because some required methods and attributes seem to be ...
fede72bari's user avatar
1 vote
1 answer
82 views

What is the difference between State Value function and Return for Markov Reward process ( MRP)?

I have been going through Stanford Lecture on RL. I see in MRP that Return function is same as State Value function. Both are getting expected sum of reward keeping discount factor in mind. Although ...
Arpit Sisodia's user avatar
0 votes
1 answer
279 views

OpenAI Gym/Gymnasium Custom Env: How should done signal be defined in a continuous task/infinite horizon problem?

I am creating a custom gym environment that should abstract a non-episodic/continuous task. Generally gym requires to return ...
94621's user avatar
  • 1
1 vote
1 answer
49 views

In Q-learning, why does Q index on both state and action?

In Q-learning, Q is an array of expected rewards for (state, action) combinations. It seems to me the same result could be achieved while slightly simplifying the algorithm, if instead of associating ...
rwallace's user avatar
  • 149
1 vote
1 answer
51 views

Why use sampling instead of the mean value for policy in Reinforcement Learning?

I'm quite new in RL and I'm currently following David Silver's course on RL. But at the same time, I also want to get hands-on, so I followed this tutorial from Gymnasium documentation: https://...
GA Faza's user avatar
  • 13
4 votes
0 answers
280 views

How Does the Reward Model in ChatGPT Calculate Losses?

Reading the InstructGPT paper(which seems to be what ChatGPT was built off of), I found this equation for the reward function. However, I'm struggling to understand how this equation is used to ...
itisyeetimetoday's user avatar

1
2 3 4 5
13