1
$\begingroup$

I have a dataset 1000 of images and corresponding segmentation masks from dermatologists. The images come in different sizes (as low as 400x600 and as large as 4Kx4K). 95% of image pixels are not targets. 5% of pixels are labeled as targets falling in one of four categories (conditions A, B, C, D). HarDNet CNN uses 352x352 input layer.

Data augmentation is done with torchvision.transforms

import torchvision.transforms as transforms
# ...
transforms.RandomRotation(90, resample=False, expand=False, center=None),
transforms.RandomVerticalFlip(p=0.5),
transforms.RandomHorizontalFlip(p=0.5),
transforms.RandomPerspective(),
transforms.RandomAffine(10, translate=None, scale=None, shear=5, resample=False),
transforms.RandomApply([transforms.CenterCrop((self.trainsize, self.trainsize))], p=0.5),    # this
# transforms.RandomApply([transforms.RandomCrop((self.trainsize, self.trainsize))], p=0.5),  # or this
transforms.Resize((self.trainsize, self.trainsize)),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225])])

Images were either supplied as is or cropped on 704x704 sections. First approach loses most of the information when the CenterCrop is applied (only 352x352 of 4Kx4K is used, 99% of the image outside of the central region removed by cropping). When CenterCrop is not applied the the texture information is lost due to loss of resolution. When I crop images to 704x704 sections I have a problem shown below:

Cropping problem

The class depends on the color of the central circle: A if red and B if blue. An image is cropped in four images (1, 2, 3, 4). There is no way for a model (or a human) to tell if upper left crop contains class A or class B because the critical information (color of central circle) is not visible. Same is true for the lower left crop.

Imbalanced data is another problem.

I've tried several ways to prepare data:

  1. Remove all images without target pixels. Split image in 704x704 segments. Problem: images tend to have only class A or class B'. Model trained on class Anever seesclass B. Because class Blooks more likeclass Athan healthy skin the model labelsclass Basclass A`.
  2. Split and feed all images to the model for training. Problem: the dataset is highly imbalanced. Only one percent of image pixels belong to the given class.
  3. For each image make one hundred of random patches 704x704. Calculate percent of the label. Include in the set with probability proportional to the percent of the labeled pixels. This leads to undersampling of small target areas. Besides, either the dataset becomes huge, or most of the unlabeled areas are excluded from the dataset. This approach oversamples large labeled areas and undersamples small target areas, see below.

enter image description here

What is the right way to consume images and masks from a folder, randomly crop an image. If the segment has a large labeled area - feed it to the model with high probability. If the segment does not have labeled pixels - feed it with low probability?

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.