Does anyone know how to get the learning rate from participant data?
I'm computing all the expected values for all trials (=200)
$$V_t(S) = V_{t-1}(S)+ \alpha \cdot \text{error }_t $$
$$(\text{error }_t = R_t − V_{t-1}(S))$$
and then computing alpha at each trial using
$$\alpha= (V_t(S) - V_{t-1}(S) )/ \text{error }_t$$
but it seems not the right way!
Suggestions?
If you still have the question I hope my answer helps you. From what I understand you want to fit a RL model to participant's data who did a decision making task. The whole process of fitting RL models to data is described in great detail in Trial by Trial analysis using RL by Nathaniel Daw.
Briefly, what you observe (your data) is just actions and rewards at every time step. If the participant is using a RL model you should expect to update the values of his actions according to:
$Q_t(a_t)=Q_t(a_{t-1})+\alpha\cdot (r_t-Q_t(a_{t-1}))$
and action selection with the Boltzmann function:
$p(a_{t}|s_{t})=\frac{e^{\beta Q(s_{t},a_{t})}}{\sum_{a'}e^{\beta Q(s_{t},a')}}$
We prefer to use a stochastic policy as it simulates better human choices and it is straightforward to create a likelihood function. The likelihood function would be the product of all the probabilities $p(a_t|s_t)$. The model parameters are $[{\alpha,\beta}]$ and these can be inferred by standard Maximum Likelihood techniques.
Finally, our hypothesis (=model) can be summarized as: A subject generates a sequence of (observed) actions. To generate these actions we assumed that uses a RL model with some parameters. We want to fit the model to the data to infer these parameters and test the suitability of the model and the cognitive/psychological links of the values of the parameters.
