I'm reading this paper about mutual information in infoGAN infoGAN_paper_link and already have the code to run it. I pretty much found code for it which is fine and dandy except for the fact that I kinda don't understand some of the code in the cost function. So, I looked at the paper to dissect it for better understanding and came across some math notation that I don't understand (pic below). The usage of the notations I'm trying to figure out are how the "~" in the Expectation subscripts, "'", and "||" symbols are used.

This is what I think the notation means.

  • "~" in the expectation subscript: the variable on the left of "~" can be any continuous value coming from whatever is to the right of the "~"
  • "'" next to the "c" in P(c'|x): I have no clue. I thought those were symbols for derivates but that makes no sense for this equation so it's def not that.
  • "||": I'm not sure. I only know of these symbols being used in Norms but that's obviously not the case here.

Screenshot of a formula from the paper screenshot from the paper

The actual code I was trying to figure out was this in PyTorch. It's the variational lower bound (mutual information term) in the cost (It's not the formulas in the screenshot above). However, I ran into the formulas in the screenshot first before getting to the formula calculations for the variational lower bound formula.

mutual_information_lower_bound = lambda c_true, mean, logvar: Normal(mean,logvar.exp()).log_prob(c_true).mean()

Thanks for the help!


1 Answer 1

  • The notation $D_\mathrm{KL}(P \| Q)$ is pretty much standard for the Kullback-Leibler divergence. There is an interesting discussion on Mathematics SE on the reasons for the fairly unusual notation used for divergences.

  • In general, $x \sim G(z, c)$ means "$x$ is a random variable distributed according to $G(z, c)$". In the subscript the author means to take the expectation with respect to the random variable $x$, supposing that $x$ has that distribution.

  • If you have a variable, such as $a$, it is not unusual to call a related variable $a'$ (usually pronounced "a prime"). In this case $c'$ is just a random variable distributed according to $P(c\mid x)$, with the name $c'$ probably meant to remind you that the variable is in some way connected to $c$. In cases where derivatives are obviously not the right thing, this is probably what a prime symbol means.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.