I am trying to understand a neuroscience article:
- Friston, Karl J., et al. "Action and behavior: a free-energy formulation." Biological cybernetics 102.3 (2010): 227-260. (DOI 10.1007/s00422-010-0364-z)
In this article, Friston gives three equations that are, as I understand him, equivalent or inter-convertertable and refer to both physical and Shannon entropy. They appear on page 231 of the article as equation (5):
The resulting expression for free-energy can be expressed in three ways (with the use of the Bayes rules and simple rearrangements):
• Energy minus entropy
• Divergence plus surprise
• Complexity minus accuracy
Mathematically, these correspond to:
The things I am struggling with at this point are:
- the meaning of the || in the 2nd and 3rd versions of the equations;
- and the negative logs.
Any help in understanding how these equations are actually what Fristen claims them to be would be greatly appreciated. For example, in the 1st equation, in what sense is the first term energy, etc?