1
$\begingroup$

I have started learning deep learning and using keras library. But I am confused as to how to take a proper estimate of the value to use for units parameter of the dense method. I came across this tip that we can take it as the average of the number of input nodes and output nodes but everywhere it says that it comes from experience. I want to know if there are things to look out for to estimate it wisely or any other things I need to know. Thanks for the help.

classifier.add(Dense(units=6, kernel_initializer="uniform", activation="relu", input_dim=11))
$\endgroup$
1
$\begingroup$

There is no known way to determine a good network structure evaluating the number of inputs or outputs. It relies on the number of training examples, batch size, number of epochs, basically, in every significant parameter of the network.

Moreover, a high number of units can introduce problems like overfitting and exploding gradient problems. On the other side, a lower number of units can cause a model to have high bias and low accuracy values. Once again, it depends on the size of data used for training.

Sadly it is trying some different values that give you the best adjustments. You may choose the combination that gives you the lowest loss and validation loss values, as well as the best accuracy for your dataset.

You could do some proportion on your number of units values, something like:

# Build the model
model = Sequential()
model.add(Dense(num_classes * 8, input_shape=(shape_value,),  activation = 'relu' )) 
model.add(Dropout(0.5))

model.add(Dense(num_classes * 4, activation = 'relu'))
model.add(Dropout(0.2))

model.add(Dense(num_classes * 2, activation = 'relu'))
model.add(Dropout(0.2))

#Output layer
model.add(Dense(num_classes, activation = 'softmax'))

The model above shows an example of a categorisation AI system. The num_classes are the number of different categories the system has to choose. For instance, in the iris dataset from Keras, we have:

  • Iris Setosa
  • Iris Versicolour
  • Iris Virginica

num_classes = 3

However, this could lead to worse results than with other random values. We need to tune the parameters to the training dataset by trial and error.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.