0
$\begingroup$

I did dozens (or probably hundreds) of tests and the best result with less total parameters(4 times or less) was a decreasing filter layers size architecture. This is a CNN for multiclass image classification, the dataset was more than 100.000 of images and each architecture was tested for at least 30 epochs, the structure was something like:

model = tf.keras.Sequential([
  layers.Conv2D(256, 3, padding='same', activation=lisht),
  layers.BatchNormalization(),
  layers.MaxPooling2D(),
  layers.Conv2D(128, 3, padding='same', activation=lisht),
  layers.BatchNormalization(),
  layers.MaxPooling2D(),
  layers.Conv2D(64, 3, padding='same', activation=lisht),
  layers.Flatten(),
  layers.Dense(512, activation=lisht),
  layers.BatchNormalization(),
  layers.Dense(3)
])

My understanding from the regular increasing structure would be that the smaller filters learn more general details, and the next ones with a bigger size will learn more complex details.So I cannot understand why the architecture mentioned above works better than its opposite (64,128,256,512).

The images are CT-scans from different types of pneumonia, where 2 classes are very alike and one is very different from these. So the "work" of the NN is to understand many characteristics from each class (CTs have many layers, and some are very different even when from the same patient). The accuracy,F1 and Kappa are approximatelly 96% for all classes using a validation dataset with 20% of the data.

$\endgroup$
2
  • $\begingroup$ What's your data like? Specialized images in any way, or just ImageNet or something? $\endgroup$
    – Ceph
    Commented Jan 7, 2022 at 12:19
  • $\begingroup$ Hi @Ceph! The images are CT-scans from different types of pneumonia, where 2 classes are very alike and one is very different from these. So the "work" of the NN is to understand many characteristics from each class (CTs have many layers, and some are very different even when from the same patient). The accuracy,F1 and Kappa are approximatelly 96% for all classes using a validation dataset with 20% of the data. Thank you! $\endgroup$
    – John
    Commented Jan 7, 2022 at 22:58

1 Answer 1

0
$\begingroup$

I think, you are trying to link a particular analysis logic to model architecture, however this is not how this not how cnn and most DL models work.

The logic behind decreasing the filter size or the network dimensionality as a general rule, is the "distillation" of characteristics that differentiate classes. In other words a representation of those characteristics is required to be packaged into a fixed length vector.

$\endgroup$
1
  • $\begingroup$ Hi @hH1sG0n3, thank you for the explanation! So, summarizing, the commonly used increasing filter size architecture would be a way to progressively learn more complexes and smaller details and connections. And the decreasing filter size architecture would be a way to just keep the most important characteristics learnt during the training to be used to discern between the classes? Thanks again! $\endgroup$
    – John
    Commented Jan 7, 2022 at 22:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.