# How to properly rotate image and labels for semantic segmentation data augmentation in Tensorflow?

What's a proper procedure for doing the image and label rotation for semantic segmentation in dataset augmentation using Tensorflow?

## Images

I have seen the function tf.contib.image.rotate(), but this function fills empty space with zeros (from docs):

Empty space due to the rotation will be filled with zeros.

I would like to fill that empty space with a different value (maybe some constant, like the dataset mean pixel). How this can be done in Tensorflow (I know that there are options in Keras Image Preprocessing, but I need TF)?

## Labels

Also, what about the labels? If I just use the same function (tf.contrib.image.rotate()), it will fill the empty space with zeros suggesting that pixel in those places belong to class with id 0 (since I have class labeled with 0). The one solution could be to put ignore label on those pixels (e.g. 255), but, again, the current function doesn't support default fill value ...

• Use different Augmenter Library; there are quite a few which are good enough.. Augmentor etc – Aditya Jan 28 '19 at 21:08
• But Augmentor cannot integrate with Tensorflow graph? I see in their docs that it integrates with Keras, but I need Tensorflow graph integration. – Antonio Jurić Jan 29 '19 at 10:01

I have found the following solution.

## Labels

Here is a snippet filling an empty space in label with ignore label.

IGNORE_LABEL = 255
label = tf.subtract(label, IGNORE_LABEL)
label = tf.contib.image.rotate(label, angle)
label = tf.add(label, IGNORE_LABEL)


This way, by subtracting IGNORE_LABEL, you shift the interval to [-255, 0]. When label (segmentation labels) is rotated, empty spaces are filled with zeros. By adding back IGNORE_LABEL, you shift interval back to [0, 255] meaning that empty space is set to IGNORE_LABEL.

## Images

Similar trick can be applied to image rotation to fill empty space with preferred value.