I am not a math expert but have a basic understanding of linear algebra, calculus and probability and I understand the math behind back propagation. Currently I am trying to learn about policy gradient algorithm, but I am having difficulty understanding the underlying math. The most commonly used cost functions in the neural network training includes a target and output. For example:

MSE:

$$E_{total} = \sum{\frac{1}{2}(target - output)^2}$$

Log Loss:

$$Error = Output(i) * (1 - Output(i)) * (Target(i) - Output(i))$$

The idea is to find parameters $$\theta$$ which reduces the distance between the target and the output.

But in policy gradient method the cost function is like this:

$$g = \mathbb E\Big[\sum R_t*\frac{(\partial)} {(\partial\theta)}ln\pi_\theta(a_t|s_t)\Big]$$

What is the target and output in policy gradient's cost function?

How does this cost function get minimized and how does it works?

In policy gradients we are interested in maximizing the expected reward. For this we assume that the expected reward is parametrized by parameters $$\theta$$ (e.g. a Neural Network. This means that in order to maximize the expected reward we need to find these parameters. In math notation: $$\theta^{\star}=\arg \max _{\theta} J(\theta) =\arg \max _{\theta} E_{\tau \sim p_{\theta}(\tau)}\left[\sum_{t} r\left(\mathbf{s}_{t}, \mathbf{a}_{t}\right)\right]$$
where $$\tau$$ is a trajectory sampled from policy $$p_\theta$$. To solve this we need gradient ascent so our parameters are updated: $$\theta = \theta + \alpha\nabla J(\theta)$$. Thus, if we compute the gradient of the expected reward we will get the proper update of our parameters towards greater expected rewards. You can take a look at Likelihood Ration and REINFORCE which explains analytically the whole optimization process.