Input shape of dataset in hybrid CNN-SVM classifier

Question

I am working on hybrid CNN-SVM for classification task, where I aim to use CNN for feature extraction and SVM for classification. So after the training of my CNN model as below:

import os
import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
import keras
from keras.layers import Dense, Conv2D, InputLayer, Flatten, MaxPool2D
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()

print('Training data: {}, {}'.format(x_train.shape, y_train.shape))
print('Test data: {}, {}'.format(x_test.shape, y_test.shape))
x_train, x_test = x_train / 255.0, x_test / 255.0
x_train = np.expand_dims(x_train, axis=3)
x_test = np.expand_dims(x_test, axis=3)
model = keras.models.Sequential([
    InputLayer(input_shape=(28, 28, 1), name='input_data'),
    Conv2D(32, 3, activation='relu'),
    Conv2D(32, 3, activation='relu'),
    MaxPool2D(pool_size=(2,2)),
    Conv2D(32, 3, activation='relu'),
    MaxPool2D(pool_size=(2,2)),
    Flatten(),
    Dense(128, activation='relu'),
    Dense(10, activation='softmax', name='output_logits')
])
model.compile(optimizer='adam',
              loss="sparse_categorical_crossentropy", 
              metrics=['accuracy'])

I used the following code to extract the last features from the last fully connected layer before softmax to be used as inputs for SVM classifier:

getFeature = K.function([model.layers[0].input, K.learning_phase()],
                        [model.layers[6].output])
exTrain = getFeature([x_train[0:5000], 0])[0]
exTest = getFeature([x_train[5000:10000], 0])[0]
y_train = y_train[0:5000].reshape(y_train[0:5000].shape[0],)             
y_test = y_train[5000:10000] 

And then I train the SVM classifier using these datatset:

from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV

parameters = {'kernel':['rbf'], 
              'C':[1, 10, 100, 1000],
              'gamma':[1e-3, 1e-4]}
clf = GridSearchCV(SVC(), parameters)
clf.fit(exTrain, y_train)
svmclf = clf.best_estimator_
svmclf.fit(exTrain, y_train)

I have the following questions:

1- Are these steps correct?

2- I have read that the shape of the The last layer should be (1,10) (for mnist classification where there are 10 classes); The last layer should have the same number of nodes as the number of classes we wish to predict for. In my case the shape of dense layer which is (1,128) which is the sahpe of exTrain the input of SVM classifier. Am confused so any clarifications please?

Regards

Answer 1

SVM is NOT needed at end.

Last dense layer of CNN network should be dense layer with 10 neurons as there can be 10 logits outputs corresponding to 10 MNIST classes.

Then apply a direct Softmax activation at end of above CNN dense layer with categorical cross entropy loss minimization to predict 10 probabilities for 10 possible digit classes.

Many many code examples of this online by googling it. You can also find jumpstart projects on MNIST classification in DL and Vision on popular code repositories like Github and others.