The paper is discussing binary classification target values only. Sometimes binary classification target values are encoded as either -1 or 1. These are the asymptotic limits of a logit-type function. Instead, it might be empirically more useful to encode binary classification target values to either -0.9 or .9.
These are a few approaches I found in the research field that combines both RNN and Reinforcement Learning that looks promising
Reinforcement learning with LSTM networks
Reinforcement learning with RNN
Hybrid RNN approach
Research paper links
A Reinforcement Learning and Recurrent Neural Network Based Dynamic User Modeling System
Supervised Reinforcement ...
After weeks trying to get into the very depth of the question as someone who's not from a maths background, neither data science, I figured out how it works internally.
This is one, raw way to make it work, and my implementation, haven't seen it anywhere implemented like this, I'm pretty sure there are more modern ways to do it, so if you want, use it with ...