I have a binary classification problem (e.g. if there is a human in the room or not) with a small dataset of images from a thermal camera. Originally, those were 7 videos, which I have converted into images of the 224 x 224 pixels. The images are of a still man sitting on a chair, two men sitting on a chair(there are very clear thermal outlines), or an empty chair(mostly a dark background).
I have about 205 images. so I have 76 no human images with label 0 (the empty chair) and 129 human images with label 1 (one man or two men).
I have used a train_test_split() to split these into the test and train datasets proportionally using stratify for an equal representation of labels 0 and 1, and then into the train and validation datasets :
from sklearn.model_selection import train_test_split X_train_images, X_test_images, y_train_labels, y_test_labels = train_test_split(all_images, all_labels_binary, stratify= all_labels_binary, random_state=0, test_size = .10) X_train_images, X_val_images, y_train_labels, y_val_labels = train_test_split(X_train_images, y_train_labels,stratify= y_train_labels, test_size=0.30)
My shapes are:
Train: X_train_images=(96, 224, 224, 3), y_train_labels=(96, 1) Validation: X_val_images=(42, 224, 224, 3), y_val_labels=(42, 1) Test: X_test_images=(21, 224, 224, 3), y_test_labels=(21, 1)
then I did a similar split to separate into a test and training set.
My model is defined as follows:
cnn_model = tf.keras.Sequential([ Conv2D(32, (5,5), padding = 'same', activation = 'relu', input_shape =(input_shape)), Conv2D(64, (3,3), padding = 'same', activation = 'relu', input_shape =(input_shape)), MaxPooling2D(2,2), Flatten(), Dense(64, activation = 'relu'), Dense(1, activation = "sigmoid"), ]) cnn_model.summary() Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= conv2d (Conv2D) (None, 224, 224, 32) 2432 _________________________________________________________________ conv2d_1 (Conv2D) (None, 224, 224, 64) 18496 _________________________________________________________________ max_pooling2d (MaxPooling2D) (None, 112, 112, 64) 0 _________________________________________________________________ flatten (Flatten) (None, 802816) 0 _________________________________________________________________ dense (Dense) (None, 64) 51380288 _________________________________________________________________ dense_1 (Dense) (None, 1) 65 ================================================================= Total params: 51,401,281 Trainable params: 51,401,281 Non-trainable params: 0
Now I am having a perfect fit with my validation and test datasets according to my accuracy and loss graphs.
Can someone explain to me why there is a perfect fit (100% accuracy), is it because I have very limited data? what can I do to make my results more meaningful? if I split my videos into more images (say every 3 seconds instead of every 10 seconds), will my results still be the same?
I wonder if try some more models like VGG16 or Resnet in keras, will my results will still be the same with no difference between the models?
Thank you for any advice.