1
$\begingroup$

I want to train a model using VGG16 to classify radio signals by their modulation typ. similar to this paper (Over the Air Deep LearningBased Radio Signal Classification) So I have built the model from scratch using Keras and I set the input shape as (2, 1024) 1024 complex points:

batch_size = 512
num_classes = 3
epochs = 100
img_rows, img_cols = 2, 1024
hf = h5py.File('ask.hdf5', 'r')

x = np.array(hf['X'][::])
y = np.array(hf['Y'][::])
if K.image_data_format() == 'channels_first':
    x = x.reshape(x.shape[0], 1, img_rows, img_cols)
    input_shape = (1, img_rows, img_cols)
else:
    x = x.reshape(x.shape[0], img_rows, img_cols, 1)
    input_shape = (img_rows, img_cols, 1)

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.15, shuffle=True)
x_training, x_val, y_training, y_val = train_test_split(x_train, y_train, test_size=0.15, shuffle=True)

model = Sequential()
model.add(Conv2D(64, kernel_size=(1, 3), activation='relu', padding='same', input_shape=input_shape))
model.add(Conv2D(64, (1, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(1, 2), strides=(1, 2)))
model.add(Conv2D(128, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(128, (1, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(1, 2), strides=(1, 2)))
model.add(Conv2D(256, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(256, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(256, (1, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(1, 2), strides=(1, 2)))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(1, 2), strides=(1, 2)))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(Conv2D(512, (1, 3), padding='same', activation='relu'))
model.add(MaxPooling2D(pool_size=(1, 2), strides=(1, 2)))
model.add(Flatten())
model.add(Dense(4096, activation='relu'))
model.add(Dense(4096, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))
model.compile(loss=keras.losses.categorical_crossentropy, optimizer='adam', metrics=['accuracy'])
callbacks = [keras.callbacks.ModelCheckpoint('model0c.h5', monitor='val_acc', verbose=0, save_best_only=True, mode='auto'), keras.callbacks.EarlyStopping(monitor='val_acc', patience=20, verbose=0, mode='auto')]
history = model.fit(x_training, y_training, batch_size=batch_size, epochs=epochs, verbose=2, validation_data=(x_val, y_val), callbacks=callbacks)

But during the training I can see that the model is not learning and the metrics are constant.

Epoch 1/100
 - 153s - loss: 1.1004 - accuracy: 0.3348 - val_loss: 1.0991 - val_accuracy: 0.3236
Epoch 2/100
 - 153s - loss: 1.0988 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 3/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0986 - val_accuracy: 0.3236
Epoch 4/100
 - 147s - loss: 1.0986 - accuracy: 0.3298 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 5/100
 - 147s - loss: 1.0987 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 6/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0989 - val_accuracy: 0.3236
Epoch 7/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0986 - val_accuracy: 0.3236
Epoch 8/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0989 - val_accuracy: 0.3236
Epoch 9/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 10/100
 - 149s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 11/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 12/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 13/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 14/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 15/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 16/100
 - 146s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0986 - val_accuracy: 0.3236
Epoch 17/100
 - 149s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 18/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236
Epoch 19/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0989 - val_accuracy: 0.3236
Epoch 20/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 21/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 22/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 23/100
 - 146s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 24/100
 - 148s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 25/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0988 - val_accuracy: 0.3236
Epoch 26/100
 - 147s - loss: 1.0986 - accuracy: 0.3374 - val_loss: 1.0987 - val_accuracy: 0.3236

What could be the problem?

$\endgroup$
1
$\begingroup$

You could possibly try varying the learning rate or initialise with different weights. Sometimes the optimiser will get stuck in certain local optimums. Alternatively try starting with pretrained weights and perform transfer learning. I found that even when used on a different domain (image to radio signals), it gives good starting accuracies

$\endgroup$
1
$\begingroup$

some more information is required to gauge why this might be happening.

  1. what is the size of your dataset.
  2. what is your learning rate? From the look at the log it seems like your lr is too low, try increasing it until you find a good starting point.
  3. Have tou tried varying other hyper-parameters?
$\endgroup$
2
  • $\begingroup$ the dataset is 19968 samples, each sample is (2, 1024) shaped. the learning rate is by default 0.001 for Adam optimizer. For the hyper-parameters, which ones should check(lr, batch-size, epochs ...)? $\endgroup$ – nechi Feb 19 '20 at 15:18
  • $\begingroup$ You should increase the learning rate to 0.1 and watch the loss/accuracy. $\endgroup$ – Aniket Feb 20 '20 at 4:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.