# How to deal with severe overfitting in a UNet Encoder/Decoder CNN in a task very similar to image translation?

I am trying to fit a UNet CNN to a task very similar to image to image translation. The input to the network is a binary matrix of size (64,256) and the output is of size (64,32). The columns represent a status of a communication channel where each entry in the column is the status of a subchannel. 1 means that the subchannel is occupied and 0 means that the subchannel is vacant. The horizontal axis represents the flow of time. So, the first column is status of the channel at time slot 1 and the second column is the status at time slot and so forth. The task is to predict the status of the channel in the next 32 time slots given the the previous 256 time slots which I treated as image to image translation. The accuracy on the training data is around 90% while the accuracy on the test is around 50%. By accuracy here, I mean the average percentage of correct entries in each image. Also, while training the validation loss increases while the loss decreases which is a clear sign of overfitting. I have tried most of the regularization techniques and also tried reducing the capacity of the model but this only reduces the training error while not improving the generalization error. Any advice or ideas? I included in the next part the learning curve for training on 1000 samples, the implementation of the network and samples from the training and test sets.

Learning curves of training on 1000 samples

3 Samples from the training set

3 Samples From the test set

Here is the implementation of the network:

def define_encoder_block(layer_in, n_filters, batchnorm=True):
# weight initialization
init = RandomNormal(stddev=0.02)
g = Conv2D(n_filters, (4,4), strides=(2,2), padding='same',
kernel_initializer=init)(layer_in)
if batchnorm:
g = BatchNormalization()(g, training=True)
# leaky relu activation
g = LeakyReLU(alpha=0.2)(g)
return g

# define a decoder block
def decoder_block(layer_in, skip_in, n_filters, filter_strides, dropout=True, skip=True):
# weight initialization
init = RandomNormal(stddev=0.02)
g = Conv2DTranspose(n_filters, (4,4), strides=filter_strides, padding='same',
kernel_initializer=init)(layer_in)
g = BatchNormalization()(g, training=True)
if dropout:
g = Dropout(0.5)(g, training=True)
if skip:
g = Concatenate()([g, skip_in])
# relu activation
g = Activation('relu')(g)
return g

# define the standalone generator model
def define_generator(image_shape=(64,256,1)):
# weight initialization
init = RandomNormal(stddev=0.02)
# image input
in_image = Input(shape=image_shape)
e1 = define_encoder_block(in_image, 64, batchnorm=False)
e2 = define_encoder_block(e1, 128)
e3 = define_encoder_block(e2, 256)
e4 = define_encoder_block(e3, 512)
e5 = define_encoder_block(e4, 512)
e6 = define_encoder_block(e5, 512)
e7 = define_encoder_block(e6, 512)
# bottleneck, no batch norm and relu
b = Conv2D(512, (4,4), strides=(2,2), padding='same', kernel_initializer=init)(e7)
b = Activation('relu')(b)
# decoder model
d1 = decoder_block(b, e7, 512, (1,2))
d2 = decoder_block(d1, e6, 512, (1,2))
d3 = decoder_block(d2, e5, 512, (2,2))
d4 = decoder_block(d3, e4, 512, (2,2), dropout=False)
d5 = decoder_block(d4, e3, 256, (2,2), dropout=False)
d6 = decoder_block(d5, e2, 128, (2,1), dropout=False, skip= False)
d7 = decoder_block(d6, e1, 64, (2,1), dropout=False,  skip= False)
# output
g = Conv2DTranspose(1, (4,4), strides=(2,1), padding='same', kernel_initializer=init)(d7)
out_image = Activation('sigmoid')(g)
# define model
model = Model(in_image, out_image)
return model