# How to deal with unbalanced data in pixelwise classification?

I'm trying to train a fully convolutional network to generate binary image showing roads from satellite photos. For a long time, I struggled to get the network to output anything but black - then I figured out the problem. I have plenty of training data, but of course most of the output pixels will be black, since roads take only small percentage of space.

I've created a toy example showing the problem:

from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv2D, MaxPooling2D
import numpy as np
import random
import sklearn.model_selection

THRESHOLD = 3
BALANCED = False

if BALANCED:
X = [[
random.randint(THRESHOLD + 1, 255) if
random.randint(0, 1) else
random.randint(0, THRESHOLD)
for _ in range(10)] for _ in range(10**4)]
Y = [[0] if x[0] <= THRESHOLD else [1] for x in X]
else:
X = [[random.randint(0, 255) for _ in range(10)] for _ in range(10**4)]
Y = [[0] if x[0] <= THRESHOLD else [1] for x in X]

X, Y = np.array(X), np.array(Y)
X_train, X_test, Y_train, Y_test = sklearn.model_selection.train_test_split(
X, Y, test_size = 0.2)

model = Sequential()
model.add(Dense(8, activation = "sigmoid", input_shape = (10, )))

model.compile(loss = "binary_crossentropy", optimizer = "adam")

model.summary()
model.fit(X_train, Y_train,
batch_size = 10, epochs = 10,
verbose = 2,
validation_data = (X_test, Y_test))

Y_res = model.predict(X_test)
Ys0 = [y[0] for x, y, y_opt in zip(X_test, Y_res, Y_test) if y_opt[0] == 0]
Ys1 = [y[0] for x, y, y_opt in zip(X_test, Y_res, Y_test) if y_opt[0] == 1]

print(np.mean(Ys0))
print(np.mean(Ys1))


Here, I artificially create data, with X being ten random bytes, and Y being boolean thresholding first byte on the value of three. That means normally only around 1/64 of all samples will be generated with y equal to 0, similarly to my original problem. There's also a second dataset, with the same decision boundary, but more evenly (class-wise) distributed data - set by changing BALANCED toggle in the code. The results confirm my suspicions.

For balanced set, actual 0's are given, on average, value of 0.025. 1's - 0.97. That's reasonable. On the unbalanced, raw dataset, 0's have mean of 0.87, and 1's - 0.98. That means the zeros are pretty much always classified as one anyway. On my real data, the effect is even worse, and the returned value is pretty much equal to percentage of black, regardless of real classification.

I tried adding class_weights = [100, 1] to fit call, but it didn't change much. Is there a good strategy for dealing with such unbalanced data? In this toy example I could probably sample the x-y pairs (e.g. keep only 5% of 1-classified data), but I don't think this would work for my full problem - since I'm using FCN, I'm putting a whole image into the network at once. There are no images with 50% roads, so I cannot possibly sample images to make roads take enough space.

I also tried, quite randomly, changing loss functions, including using one for categorical data and one-hot encoding expected outputs (i.e. [1, 0] and [0, 1] instead of 0 and 1). This did not fix anything either.

• I think that your artificial example does not quite correspond to your real task. The roads are continuous lines. However, the zeros in your example are isolated. This makes the problem more difficult. You real task is the task of line detection. There are many methods in image processing for that. Maybe some of them use CNN. I think it is worth changing you artificial example, make it contain continuous lines with intersections, etc just like roads. This will make the task easier which will increase a chance that someone comes up with a solution. May 23, 2018 at 2:03