I am trying to classify 6 classes time-frequency domain signal (STFT spectrogram) with a size of 3601x217 pixels. Assume that for each classes have 70 training samples, 20 validation samples, and 10 test samples. Each sample is a PNG image of size 3601x217x1 (grayscale), while in the future I am thinking to expand this into 3601x217x4 (3 color channels + alpha). But right now, I am focusing on grayscale first.
After trying pure LSTM, I found out that the model overfits quickly (training accuracy > 90%, but val_acc stucks at 20%). ow I'd like to try out Time Distributed CNN+LSTM.
Following is the code:
img_width, img_height = 3601,217
train_data_dir = 'sensor1/training'
validation_data_dir = 'sensor1/validation'
num_classes = 6
nb_train_samples = 70
nb_validation_samples = 20
epochs = 50
batch_size = 5
if K.image_data_format() == 'channels_first':
input_shape = (3, img_width, img_height)
else:
input_shape = (img_width, img_height, 3)
model = Sequential()
model.add(TimeDistributed(Conv2D(16, (3,3), padding='same', strides=(2,2), activation='relu', input_shape = input_shape)))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(Dropout(0.5))
model.add(TimeDistributed(Conv2D(32, (3,3), padding='same', strides=(2,2), activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(Dropout(0.5))
model.add(TimeDistributed(Conv2D(64, (3,3), padding='same', strides=(2,2), activation='relu')))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))
model.add(Dropout(0.5))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(units=128, return_sequences=False))
model.add(LSTM(units=64, return_sequences=False))
model.add(Dense(32))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
train_datagen = ImageDataGenerator(rescale = 1. / 255)
test_datagen = ImageDataGenerator(rescale = 1. / 255)
train_generator = train_datagen.flow_from_directory(
train_data_dir, target_size=(img_width, img_height),
batch_size=batch_size, color_mode='grayscale')
validation_generator = test_datagen.flow_from_directory(
validation_data_dir, target_size=(img_width, img_height),
batch_size=batch_size, color_mode='grayscale')
model.fit_generator(
train_generator, steps_per_epoch=nb_train_samples // batch_size,
epochs=epochs, callbacks=[plot_losses],
validation_data=validation_generator,
validation_steps=nb_validation_samples // batch_size)
The code gives following error:
File "C:\ProgramData\Anaconda3\lib\site-packages\keras\engine\training.py", line 496, in _make_train_function raise RuntimeError('You must compile your model before using it.')
RuntimeError: You must compile your model before using it.
What's wrong with the code? I actually compiled the model before calling model.fit_generator:
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])