0
$\begingroup$

I have been trying to understand the below code (written in Tensorflow), but unable to do so as I am not very proficient with Tensorflow. I am looking for some help. Below is the code:

    if model == 'mse':
        rec_loss = tf.losses.mean_squared_error(img, img_rec)
        kld_loss = -tf.reduce_mean(0.5 * (1 + z_log_sigma_sq - z_mu ** 2 - tf.exp(z_log_sigma_sq)))
    else:
        if model == 'gaussian':
            log_sigma = tf.Variable(0.0, trainable=False)
        elif model == 'sigma':
            log_sigma = tf.Variable(0.0, trainable=True)
        elif model == 'optimal':
            log_sigma = tf.log(tf.sqrt(tf.reduce_mean((img - img_rec) ** 2, [0, 1, 2, 3], keepdims=True)))

        rec_loss = tf.reduce_sum(gaussian_nll(img_rec, log_sigma, img))

Where the gaussian_nll function is defined below:

def gaussian_nll(mu, log_sigma, x):
    return 0.5 * ((x - mu) / tf.exp(log_sigma)) ** 2 + log_sigma + 0.5 * np.log(2 * np.pi)

I have these questions:

  1. What is the difference between log_sigma under model == 'gaussian' and under model == 'sigma'? (only difference in the definition of the variable is the flag trainable)
  2. What is the difference between the tf.losses.mean_squared_error under the model=='mse' and log_sigma under model == 'optimal'? Aren't they both similar?

They are correct, but I am unable to correctly read it. Any help regarding the code would be greatly appreciated. For those of you who wants to take a look at the full code on github, here is the link github page (the above piece of code appears between lines 34 and 44)

Thanks once again for your help.

$\endgroup$
1
$\begingroup$

gaussian is different from sigma, because sigma is trainable while gaussian is not. It implies that the value of sigma can be optimized but not of gaussian.

optimal is different from mse as

  • in the case of mse, the final loss is computed as MSE.
  • but in the case of optimal, we use gaussian as the loss function, with MSE acting as the variance of the gaussian kernel.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.