Take the Horde paper, there is this GVF, General Value Function Approximators which have a pseudo termination function and a pseudo reward. If I understand correctly, multiple demons with different behavioral policies, each of them having a GVF, spawned and off-policy learning takes place with GQ probability distribution. What is the pseudo termination function here? Is it a state?
Similar to this, there is this "goal" in UVFA paper. Take this paragraph for example, at the end of the second page:
A two-stream architecture, on the other hand, assumes that the problem has a factorized structure and computes its output from two components φ : S → R and ψ : G → R, that both map into an n-dimensional vector space of embeddings and output function h : R × R → R that maps two such embedding vectors to a single scalar regression output.
The novelty of this paper is that the value function generalizes not just over states but also goals, as introduced in the abstract. But I'm having hard time understanding the goal definition here. Again, is it a state that an agent wants to achieve?
So, can anyone give me a clue about the meaning behind these two concepts?