I have the following architecture of my network:
def net_one(message): weight1 = np.random.normal(loc=0.0, scale=0.01, size=[16, 16]) init1 = tf.constant_initializer(weight1) out1 = tf.layers.dense(inputs=message, units=16, activation=tf.nn.relu, kernel_initializer=init1) weight2 = np.random.normal(loc=0.0, scale=0.01, size=[16, 7]) init2 = tf.constant_initializer(weight2) out2 = tf.layers.dense(inputs=out1, units=7, activation=None, kernel_initializer=init2) return out2
Now as the output of the network is linear (
None in tensorflow corresponds to a linear activation function), the output is unbounded. I need the square of the
2-norm of the output to be a constant,
n (for energy constraint purposes). I do not want to use
tanh as they hamper the performance. I tried the following:
code = net_one(input_bits) code = code * tf.sqrt(n) / tf.linalg.norm(code)
I have two questions:
- Does it achieve what I expect it to achieve?
- Is there any better way (if this is indeed right) or any alternate way to achieve this?