face landmark detection cnn loss not converging in tensorflow

Question

I am trying to build face landmark detection model using simple regression.I used celeba dataset which has 5 points hence 10 output units.I used grayscale and normalized image as input. Here is my model

    self.model = models.Sequential()
    self.model.add(layers.Conv2D(32, (4, 4), activation='relu', input_shape=(218, 178,1)))#l1
    self.model.add(layers.Conv2D(64, (4, 4), activation='relu'))#l2
    self.model.add(layers.Conv2D(32, (4, 4), activation='relu'))#l3
    self.model.add(layers.Conv2D(16, (4, 4), activation='relu'))#l4
    self.model.add(layers.Conv2D(16, (3, 3), activation='relu'))
    self.model.add(layers.Conv2D(16, (3, 3), activation='relu'))
    self.model.add(layers.Conv2D(32, (3, 3), activation='relu'))
    self.model.add(layers.Conv2D(64, (3, 3), activation='relu'))
    self.model.add(layers.Conv2D(128, (3, 3), activation='relu'))
    self.model.add(layers.AveragePooling2D((3, 3)))
    self.model.add(layers.Flatten())
    self.model.add(layers.Dense(16))
    self.model.add(layers.Dense(10)) 

Here is my loss function

model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.001),loss='mse')

Here loss is stuck around 18 and doesn't go below that. I tried various configuration of CNN architecture like adding and removing layers and I also tried with changing learning rate but no use.

Please anyone point me in right direction. How can I debug this network. (For this I only used first 100 images of dataset)

Answer 1

Use a standard pretrained network like ResNet, Effnet or even VGG16 for feature extraction and put a regression layer in the end.

Now, talking about your network. FIrst of all, the image size is not square, make it! Don't use even valued kernels (3 and 5 are mostly used). Number of filters increase with depth, yours don't follow a pattern.

Lastly, 100 images are nothing for these behemoths, give it more data.

Better stick to a pretrained model.