Tensorflow 2.0 - Layer with fixed input

Question

I'm trying to use Tensorflow to optimize a few variables to be used in a KNN algorithm, however, I'm running into an issue where I'm unable to have a layer work properly if it is not connected to an Input layer.

What I'm trying to do below is pass in [[1]] as a static tensor, and then using the Lambda layer to coerce the weights into a usable shape. Once trained, I would just retrieve the weight value from the first Dense layer.

ones = tf.ones(shape=(1,1)) # trying to use this to take the place of dynamic inputs

theta_layer = layers.Dense(1, activation="linear", use_bias=False, trainable=True)(ones)
theta_layer = layers.Lambda(lambda x: tf.ones(shape=(self.batch_size,1)) * x)(theta_layer)
print(theta_layer)
# tf.Tensor(
# [[-1.151663]
# [-1.151663]
# [-1.151663]], shape=(3, 1), dtype=float32)

concat_layer = layers.Concatenate()([theta_layer, features_input])

model = models.Model(inputs=features_input, outputs=concat_layer)
model.compile(optimizer="adam", loss="mse") #, metrics=["mae"])#, bias_regularizer=None)

The problem with the above strategy is that theta_layer does not show up in model.summary() and the weights don't show up in model.get_weights().

Also, when I try to directly set theta_layer as the model output, I get this message:

AttributeError: Tensor.op is meaningless when eager execution is enabled.

This is probably the giveaway but frankly I'm just not knowledgeable enough about Tensorflow.

I know that I can fix this by passing in a separate tf.ones input to the model (not just into specific layers), but it would complicate the code seemingly unnecessarily (I have to do this for multiple layers/variables). Is there a way that I can alter the tf.ones passed to the layer or the alter the layer itself to eliminate this issue?

Answer 1

This feels like a bit of a hack, but I was able to infer something from an answer on another question: https://stackoverflow.com/a/46466275/6182971

It works if I change the first line above from:

ones = tf.ones(shape=(1,1))

to:

ones = layers.Lambda(lambda x: tf.ones(shape=(1,1)))(features_input)

Even though the Lambda layer is returning a constant, passing in features_input, which is the main training data connects the tf.ones constant to the network inputs, which seems to be sufficient.