1
$\begingroup$

I had to use tf.data.Dataset explicitly to get a tf.keras.experimental.LinearModel fitted without weights (i.e. with bias only), but I think I have it set up right:

import tensorflow as tf
import numpy as np

batch = 1000
dataset = tf.data.Dataset.from_tensor_slices(
    (
        np.ones(shape=(batch, 0), dtype=float),
        np.ones(shape=(batch,), dtype=float),
    )
)

Note that the inputs are empty, because their shape has a zero. Okay, let's try to estimate bias (it should converge to 1.0).

lm = tf.keras.experimental.LinearModel(1)
lm.compile(
    optimizer=tf.keras.optimizers.SGD(),
    loss=tf.keras.losses.MeanSquaredError(),
)
lm.fit(dataset.batch(batch), epochs=100, verbose=0)
lm.get_weights()
[array([], shape=(0, 1), dtype=float32), array([1.9999833], dtype=float32)]

Huh, that's not so good. Maybe it's just really slow.

lm.fit(dataset.batch(batch), epochs=100, verbose=0)
lm.get_weights()
[array([], shape=(0, 1), dtype=float32), array([5.9999557], dtype=float32)]

Wait, it's getting worse? Please help me understand what's going on, and show me how you can poke the model to see it. Thanks!

$\endgroup$
1
  • $\begingroup$ In contrast, I've found that using tf.ones as input to a tf.keras.Model with output given by tf.keras.layers.Dense(1, use_bias=False) works just great. $\endgroup$
    – Ian
    Mar 9, 2022 at 18:48

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.