I am classifying aerial imagery that is tiled into 256x256 tiles using Keras and TensorFlow. The model splits the training data (i.e. the 256x256 image tiles making up the study area) into 70% training data and 30% validation data. A sequential model is used followed by an image data generator. Lastly, a fit generator is used to fit the model to the data. The model is then saved to h5 format to be used to predict classes with other imagery in different study areas.

When I run the model using the 70%/30% training/validation split, the predictions on the validation images work great with increasingly higher accuracies and steadily decreasing loss per epoch. Additionally, when I visualize the predictions (i.e. probability arrays) by joining the probability arrays to vector polygons representing the tile boundaries, the classified results look very good.

My problem is when I use the saved h5 model to make predictions on new imagery--the results are nonsensical and appear random for each tile. It is as if the probability arrays are being shuffled randomly such that when I join the results to the vector image boundary tiles, the results look totally random. How can I resolve this issue?

Here is relevant portions of the code used to train the model:

base_model = applications.VGG16(weights='imagenet', include_top=False, input_shape=(img_rows, img_cols, img_channel))

add_model = Sequential()
add_model.add(Flatten(input_shape=base_model.output_shape[1:]))
add_model.add(Dense(256, activation='relu'))
add_model.add(Dense(n_classes, activation='sigmoid')) # n classes

model = Model(inputs=base_model.input, outputs=add_model(base_model.output))
model.compile(loss='binary_crossentropy', optimizer=optimizers.SGD(lr=1e-4, momentum=0.9),
              metrics=['accuracy'])

######################

batch_size = 32
epochs = 50

print('Running the image data generator...')
train_datagen = ImageDataGenerator(
        rotation_range=30, 
        width_shift_range=0.1,
        height_shift_range=0.1, 
        horizontal_flip=True)
train_datagen.fit(x_train)

print('Fitting the model...')
history = model.fit_generator(
    train_datagen.flow(x_train, y_train, batch_size=batch_size),
    steps_per_epoch=x_train.shape[0] // batch_size,
    epochs=epochs,
    #validation_data=(x_valid, y_valid),
    #callbacks=[ModelCheckpoint(model_checkpoint, monitor='val_acc', save_best_only=True)]
)

######################

## Predict
#print('Predicting...')
#p_valid = model.predict(x_valid, batch_size=128)

## Write predictions to csv
#print('Saving predictions to CSV...')
#df = pd.DataFrame(p_valid)
#df['image'] = split + 1 + df.index 
#df.to_csv(out_csv, index=False, header=False)

""" 
Save model, including these details:
-the architecture of the model, allowing to re-create the model
-the weights of the model
-the training configuration (loss, optimizer)
-the state of the optimizer, allowing to resume training exactly where you left off.
"""
print("Saving model")
model.save("/vgg16-model-50epochs.h5")

print('Processing complete.')

And the following script uses the saved model from above to make predictions on test imagery from a different study area. Note, there is no 70/30 training/validation split in the final training run above--I simply use 100% of the tiles to train the model, which I then save and reuse in the following script:

import glob, os, time
import cv2
import numpy as np
import pandas as pd

from keras.models import load_model
#from keras.models import model_from_json

# Path to the input tiles which will be used to predict classes
inws = '/image-directory-for-another-study-area'
tiles = glob.glob(os.path.join(inws, '*.tif'))

# h5 file from trained model
in_h5 = "/vgg16-model-50epochs.h5"

# Output model predictions in csv format
out_csv = '/new-predictions.csv'

# Read images and convert to numpy array
x_test = np.array([cv2.imread(tile) for tile in tiles], np.float16) / 255.

print('Loading existing model...')
loaded_model = load_model(in_h5)

print("Predicting on image tiles...")
predictions = loaded_model.predict(x_test, batch_size=128)

# Save to csv
df = pd.DataFrame(predictions)
df['image'] = df.index + 1
df.to_csv(out_csv, index=False, header=False)
print("Predictions saved to disk: {0}".format(out_csv))

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.