0
$\begingroup$

I'm trying to use Tensorflow to optimize a few variables to be used in a KNN algorithm, however, I'm running into an issue where I'm unable to have a layer work properly if it is not connected to an Input layer.

What I'm trying to do below is pass in [[1]] as a static tensor, and then using the Lambda layer to coerce the weights into a usable shape. Once trained, I would just retrieve the weight value from the first Dense layer.

ones = tf.ones(shape=(1,1)) # trying to use this to take the place of dynamic inputs

theta_layer = layers.Dense(1, activation="linear", use_bias=False, trainable=True)(ones)
theta_layer = layers.Lambda(lambda x: tf.ones(shape=(self.batch_size,1)) * x)(theta_layer)
print(theta_layer)
# tf.Tensor(
# [[-1.151663]
# [-1.151663]
# [-1.151663]], shape=(3, 1), dtype=float32)

concat_layer = layers.Concatenate()([theta_layer, features_input])

model = models.Model(inputs=features_input, outputs=concat_layer)
model.compile(optimizer="adam", loss="mse") #, metrics=["mae"])#, bias_regularizer=None)

The problem with the above strategy is that theta_layer does not show up in model.summary() and the weights don't show up in model.get_weights().

Also, when I try to directly set theta_layer as the model output, I get this message:

AttributeError: Tensor.op is meaningless when eager execution is enabled.

This is probably the giveaway but frankly I'm just not knowledgeable enough about Tensorflow.

I know that I can fix this by passing in a separate tf.ones input to the model (not just into specific layers), but it would complicate the code seemingly unnecessarily (I have to do this for multiple layers/variables). Is there a way that I can alter the tf.ones passed to the layer or the alter the layer itself to eliminate this issue?

$\endgroup$
0
$\begingroup$

This feels like a bit of a hack, but I was able to infer something from an answer on another question: https://stackoverflow.com/a/46466275/6182971

It works if I change the first line above from:

ones = tf.ones(shape=(1,1))

to:

ones = layers.Lambda(lambda x: tf.ones(shape=(1,1)))(features_input)

Even though the Lambda layer is returning a constant, passing in features_input, which is the main training data connects the tf.ones constant to the network inputs, which seems to be sufficient.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.