Low accuracy using Multi Layer perceptron network on CIFAR10 dataset?

Question

I trained Muti layer perceptrons using keras to classify cifar-10 dataset the results I got shows that there is something wrong in the code ,because all the epoch are identical here is the code : In this I flatten the data from 32x32x3 to 3072 array.

   #nn layers 
import numpy
from keras.datasets import cifar10
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from matplotlib import pyplot
from scipy.misc import toimage

#load data
(Xt,Yt),(Xts,Yts)=cifar10.load_data()

num_pix=Xt.shape[1]*Xt.shape[2]*Xt.shape[3]
X_tr=Xt.reshape(-1,3072).astype('float32')
X_ts=Xts.reshape(-1,3072).astype('float32')
#feature scaling
X_tr= X_tr / 255
X_ts = X_ts / 255
#one hot encoding
from keras.utils import np_utils
Ytr=Yt.reshape(-1,50000)
Yt=np_utils.to_categorical(Ytr)
Yts=Yts.reshape(-1,10000)
Yts=np_utils.to_categorical(Yts)
num_class=Yts.shape[1]


def base_model():
    model=Sequential()
    #build layers Dense(10, init="normal", activation="relu"
    model.add(Dense(num_pix,input_dim=num_pix,activation='relu'))#input layer
    model.add(Dense(num_class,activation='softmax'))#output layer
    #compile model
    model.compile(loss='squared_hinge',optimizer='adam',metrics=['accuracy'])
    return model
    #build the model
model=base_model()
#Fit model
model.fit(X_tr,Yt,validation_data=(X_ts,Yts),epochs=10,batch_size=200,verbose=2)
#Final assesment
score=model.evaluate(X_ts,Yts,verbose=0)
print("Baseline Error: %.2f%%" % (100-score[1]*100))

  Results :

 Train on 50000 samples, validate on 10000 samples
    Epoch 1/10
    134s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 2/10
    97s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 3/10
    99s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 4/10
    96s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 5/10
    96s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 6/10
    96s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 7/10
    96s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 8/10
    95s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 9/10
    95s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Epoch 10/10
    95s - loss: 0.9900 - acc: 0.1000 - val_loss: 0.9900 - val_acc: 0.1000
    Baseline Error: 90.00%

Any help will be appreciated.can my model be overfitted?

Answer 1

CIFAR10 has 10 class label. So by random guessing, you should achieve an accuracy of 10%. And this is what you are getting. This means your algorithm is not learning at all. The most common problem causes this is your learning rate. Reduce your learning rate by replacing your line,

model.fit(X_tr,Yt,validation_data=(X_ts,Yts),epochs=10,batch_size=200,verbose=2)

with

optimizer = keras.optimizers.Adam(lr=1e-4)
model.compile(loss='squared_hinge',optimizer=optimizer,metrics=['accuracy'])

I just did a test run, it is learning slowly.