I am reading gail implementation code in openai baselines. they compute bernoulli entropy as one of the loss in adversary network loss function.

In their code, they implement bernoulli entropy as this:

def logsigmoid(a):
'''Equivalent to tf.log(tf.sigmoid(a))'''
return -tf.nn.softplus(-a)

def logit_bernoulli_entropy(logits):
ent = (1.-tf.nn.sigmoid(logits))*logits - logsigmoid(logits)
return ent

also there is a reference of another openai implementation, it's the same code but I can't see any explanation to it.

I checked that the equation to compute bernoulli entropy is:

$ -p\log p - (1-p)\log(1-p)$

I think the later equation is the right way to compute bernoulli entropy, but the first one should be right too as it's written in openai's implementationn. I can't see any similarity, is there any relationship between these two expression?


Good observation, and yes, they are in fact equivalent ways of computing the entropy of a bernoulli random variable.

To begin, you have to notice that in the openai code, we do not have the value of $ p $ passed to the function, instead we have $ logit(p) $ which is defined to be:

$ logit(p) = log\frac{p}{1-p} = log(p) - log(1-p)$

Also, two formula to keep in mind. The first is the sigmoid function which is:

$ sigmoid(x) = \frac{1}{1+e^{-x}} $

And the second is the sigmoid of the logit which is:

$ sigmoid(logit(p)) = \frac{1}{1+\frac{1-p}{p}} = p$

(i.e. the inverse of the logit function is the sigmoid function)

Now we are ready to go from the first equation (used in the code) to the second equation (the generic equation to compute the bernoulli entropy)

$ entropy = (1 - sigmoid(logit(p)))*logit(p) - log(sigmoid(logit(p))) $

$ = (1 - p)*logit(p) - log(p) = (1 - p)*(log(p)-log(1-p)) - log(p) $

$ = (1 - p)*log(p) - (1-p)*log(1-p) - log(p) $

$ = -p*log(p) - (1-p)*log(1-p) = entropy $

Thus the formula used in the code is equivalent to the second formula you provided.

  • $\begingroup$ Quit exhaustive answer, but I'm a further confusion. I noticed in the openai code that they define logits with logits = tf.concat([generator_logits, expert_logits], 0) , how $logit(p)=log(p)-log(1-p)$ comes? And by the way, as the inverse of logit function is sigmoid function, why can't use sigmoid as activation function in the last layer? I noticed that there are direct usage of logits so they have to use tf.identity as the activation of last layer. does this kinds of logits has any meaning? $\endgroup$ – Yang Gaoguang May 1 '20 at 16:01
  • $\begingroup$ Sure, the concatenation just means that they are applying all the calculations in my answer unilaterally on both variables. As to where my calculations come from, whenever anybody refers to a logit they are referring to this agreed upon definition link. And finally, as to where they do that calculation, they do it at some point inside graph and the model tries to reproduce the value that a logit would obtain. It has a lot of theoretical concepts built into it so don't feel frustrated if it's still confusing. $\endgroup$ – A Kareem May 1 '20 at 16:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.