0
$\begingroup$

I am building a TF CNN model that takes a picture as input and has 3 outputs (multitask learning). On one of the output layers, I would like to output a distribution object, tfp.distributions.Normal, from which I can take the mean and sttdv to figure out confidence intervals as in this example Case2. The problem is that my model does not output a tensorflow_probability.python.util.deferred_tensor.Normal type of object as in the model. Therefore, in the output I cannot call .mean() or .sttdv(). Instead it only outputs a numpy value. What am I doing wrong? How can I make my model output a distribution type object from which I can call .mean() and .sttdv()?

base_model
...
output1 = Dense(16, activation='relu')(base_model)
output1 = Dense(8, activation='relu')(output1)
output1 = Dense(1, activation='sigmoid')(output1) 

...
output2 = Dense(32, activation='relu')(base_model)
output2 = Dense(16, activation='relu')(output2)
output2 = Dense(5, activation='softmax')(output2)

...
dist_output3 = Dense(128, activation='relu')(base_model)
dist_output3 = Dense(128, activation='relu')(output3)
dist_output3 = Dense(2)(output3)
dist_output3 = tfp.layers.DistributionLambda(lambda t: tfd.Normal(loc=t[..., :1],
                                                               scale=1e-3 + tf.math.softplus(0.05 *t[...,1:])))(dist_output3)

model = Model(inputs=base_model.layers[0].input, 
              outputs=[output1, output2, dist_output3])

negloglik = lambda y, p_y: - p_y.log_prob(y)

model.compile(optimizer = 'adam', 
              loss =['binary_crossentropy', 'categorical_crossentropy', negloglik],
              metrics=['accuracy', 'mse'])

history = model.fit_generator( 
    train_generator,
    validation_data=validate_generator,
    epochs=100,
)

...
predictions = model(test_generator)
len(predictions)
>>> 3

predictions[0][0]
>>> array([0.08734298723])

predictions[1][0]
>>> array([0.26107603, 0.1662454 , 0.24182138, 0.09926344, 0.23159383])

predictions[2][0]
>>> array([0.9827457034]) <-- I am expecting the output of this to be something like array([0.9827457034, 0.246534654]) as in array([<mean>, <std>])
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.