# Problem in the parameter size in constraint parameter no hidden netwrok

I was trying to implement a no hidden layer model for classification with the constraint that all the weights have the absolute value $$1$$, As follows

import tensorflow as tf
from tensorflow.keras.constraints import max_norm
from tensorflow.keras.layers import Dense

model = tf.keras.models.Sequential()
model.compile(loss = 'mse',metrics='accuracy')
model.fit(X_train,y_train,batch_size= 100, epochs=30)


Here the input has four features, output has two feature with 0 bias. The training phase converges with very good accuracy. So there should be $$4*2 = 8$$ parameters in the model. But

model.get_weights().shapemodel.get_weights().shape


has returned a list which is

[array([[ 0.33709845, -0.55206317, -0.53412324, -0.15946062],
[ 0.03901154,  0.11299153,  0.0941186 , -0.31381565],
[ 0.69983363, -0.06287932, -0.14378403,  0.22027704],
[-0.6285503 ,  0.13892443,  0.26558703, -0.42299727]],
dtype=float32),
array([ 0.30418202,  0.29648915, -0.31335464,  0.3056269 ], dtype=float32),
array([[ 1.1184788 , -0.24037583],
[ 0.7782192 , -0.11152704],
[-0.01809523,  0.38015074],
[ 0.3689365 , -0.7091095 ]], dtype=float32)]


The parameter size does not look a $$4*2$$ output. Seems like $$4*4*2$$ is the shape I got. Please help me to know where is the problem in my code and how to make the shape $$4*2$$ keeping the weight constraint. Also the third array has one weight $$> 1$$, violate the constraints. Though the model is trained well.

The model you have made actually has a hidden layer, i.e. the 4 neuron layer is actually a hidden layer

The number of parameters is affected by the input size you put in

eg:

model = tf.keras.models.Sequential()
model.compile(loss = 'mse',metrics='accuracy')
model.fit([[1,2,3,4]],[[1,0]])  ## the input shape is (4,)
model.summary()

Model: "sequential_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_5 (Dense)              (None, 4)                 20
_________________________________________________________________
dense_6 (Dense)              (None, 2)                 8
=================================================================
Total params: 28
Trainable params: 28
Non-trainable params: 0
_________________________________________________________________


If you want to create a model with input layer of size 4 you should instead

model = tf.keras.models.Sequential()
model.compile(loss = 'mse',metrics='accuracy')
model.summary()


which gives the desired model:

Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_7 (Dense)              (None, 2)                 8
=================================================================
Total params: 8
Trainable params: 8
Non-trainable params: 0
_________________________________________________________________


The number of trainable parameters i.e. the weights is equal to the total params in model.summary()

• Can you please set the parameter constrained to have maximum norm value 1, Jun 27 '21 at 18:49
• does model.add(Dense(2,activation='sigmoid',use_bias=False, kernel_constraint=max_norm(1))) as the last layer solve the problem? Jun 27 '21 at 18:55