There are many papers out there that deal with neural networks and RL. This blog will give a very good insight on a Policy Gradient Network: Deep RL with PG

Now for your question. You really need to be familiar on how we train a neural network. A simple one for classification. If you check the derivations and how the weights are getting updated things will be very clear to you on how you can implement the above.

I will describe it to you as simple as possible so you get the link. A Neural Network in a very broad sense consists of nested functions. The function that contains all the others is the one at your output layer. In case of Stochastic Policy Gradients this is your Boltzmann function. So your output layer will take all the previous layer's outputs and will pass them through the Boltzmann function. You can see from the paper that these equations are from, that the Boltzmann function already satisfies that condition. However, in the case of NN the parametrization comes from all previous layers.

The blog link I sent you describes a very nice and simple example of a vanilla Policy Gradient with NN (REINFORCE algorithm). By using the code (plus your understanding on feedforward networks) you will see that the gradients are multiplied by the reward. It is a very good exercise!

For Actor-Critic, you need in general a network performing PG (stochastic or deterministic) -- you Actor -- and a network that will give you the reward signal (like the simple case in the blog. However, for various reasons, instead of the actual reward we use another network that estimates the reward by performing Q-learning as in Deep-Q learning (minimizing square error between estimated reward and true reward).

Hope this helps!

There are many papers out there that deal with neural networks and RL. This blog will give a very good insight on a Policy Gradient Network: Deep RL with PG

Now for your question. You really need to be familiar on how we train a neural network. A simple one for classification. If you check the derivations and how the weights are getting updated things will be very clear to you on how you can implement the above.

I will describe it to you as simple as possible so you get the link. A Neural Network in a very broad sense consists of nested functions. The function that contains all the others is the one at your output layer. In case of Stochastic Policy Gradients this is your Boltzmann function. So your output layer will take all the previous layer's outputs and will pass them through the Boltzmann function. In the case of NN the parametrization comes from all previous layers.

The blog link I sent you describes a very nice and simple example of a vanilla Policy Gradient with NN (REINFORCE algorithm). By using the code (plus your understanding on feedforward networks) you will see that the gradients are multiplied by the reward. It is a very good exercise!

For Actor-Critic, you need in general a network performing PG (stochastic or deterministic) -- you Actor -- and a network that will give you the reward signal (like the simple case in the blog. However, for various reasons, instead of the actual reward we use another network that estimates the reward by performing Q-learning as in Deep-Q learning (minimizing square error between estimated reward and true reward).

Hope this helps!

There are many papers out there that deal with neural networks and RL. This blog will give a very good insight on a Policy Gradient Network: Deep RL with PG

Now for your question. You really need to be familiar on how we train a neural network. A simple one for classification. If you check the derivations and how the weights are getting update things will be very clear to you on how you can implement the above.

I will describe it to you as simple as possible so you get the link. A Neural Network in a very broad sense consists of nested functions. The function that contains all the others is the one at your output layer. In case of Stochastic Policy Gradients this is your Boltzmann function. So your output layer will take all the previous layer's outputs and will pass them through the Boltzmann function. You can see from the paper that these equations are from, that the Boltzmann function already satisfies that condition. However, in the case of NN the parametrization comes from all previous layers.

The blog link I sent you describe a very nice and simple example of a simple Policy Gradient NN. By using the code (plus your understanding on feedforward networks) you will see that the gradients are multiplied by the reward. It is a very good exercise!

For Actor-Critic, you need in general a network performing PG (stochastic or deterministic) -- you Actor -- and a network that will give you the reward signal (like the simple case in the blog. However, for various reasons, instead of the actual reward we use another network that estimates the reward by performing Q-learning as in Deep-Q learning (minimizing square error between estimated reward and true reward).

Hope this helps!

There are many papers out there that deal with neural networks and RL. This blog will give a very good insight on a Policy Gradient Network: Deep RL with PG

Now for your question. You really need to be familiar on how we train a neural network. A simple one for classification. If you check the derivations and how the weights are getting updated things will be very clear to you on how you can implement the above.

I will describe it to you as simple as possible so you get the link. A Neural Network in a very broad sense consists of nested functions. The function that contains all the others is the one at your output layer. In case of Stochastic Policy Gradients this is your Boltzmann function. So your output layer will take all the previous layer's outputs and will pass them through the Boltzmann function. You can see from the paper that these equations are from, that the Boltzmann function already satisfies that condition. However, in the case of NN the parametrization comes from all previous layers.

The blog link I sent you describes a very nice and simple example of a vanilla Policy Gradient with NN (REINFORCE algorithm). By using the code (plus your understanding on feedforward networks) you will see that the gradients are multiplied by the reward. It is a very good exercise!

For Actor-Critic, you need in general a network performing PG (stochastic or deterministic) -- you Actor -- and a network that will give you the reward signal (like the simple case in the blog. However, for various reasons, instead of the actual reward we use another network that estimates the reward by performing Q-learning as in Deep-Q learning (minimizing square error between estimated reward and true reward).

Hope this helps!