How to specify output_shape parameter in Lambda layer in Keras

Question

I don't understand how to specify the output_shape parameter in the Lambda layer in Keras/Tensorflow. The documentation says:

output_shape: Expected output shape from function. This argument can be inferred if not explicitly provided. Can be a tuple or function. If a tuple, it only specifies the first dimension onward; sample dimension is assumed either the same as the input: output_shape = (input_shape[0], ) + output_shape or, the input is None and the sample dimension is also None: output_shape = (None, ) + output_shape

If we use a tuple how should I interpret these two expressions?

output_shape = (input_shape[0], ) + output_shape

and

output_shape = (None, ) + output_shape

Answer 1

Let's say you pass in output_shape as a tuple (50, 50, 10) where we can call the values (height, width, channels)` to the lambda layer:

your_layer = tf.keras.layers.Lambda(lambda x: x, output_shape=(50, 50, 3))

The part of the documentation:

If a tuple, it only specifies the first dimension onward;

means that the batch dimensions itself is simple carried forward, unchanged.

If you have e.g. batch_size=3 during training, the incoming tensor to your_layer might be (3, n, p, q), where n p and q could be anything, but the layer is expected to produce a shape (3, 50, 50, 10). So the 0 dimension remains unchanged, and we have concatenated it with your output_shape:

(3,) + (50, 50, 10) -> (3, 50, 50, 10)

This corresponds to the expression: output_shape = (input_shape[0], ) + output_shape, so we see that input_shape is the true shape of the incoming batch tensor during training, as we only took the batch dimension to produce the layer's outgoing batch tensor.

For the second expression it is really just the same thing, but if you haven't provided a batch shape, Tensorflow & Keras represent that as something that could be anything, and store it as None. So in that case you get:

(None,) + (50, 50, 10) -> (None, 50, 50, 10)