I have a binary classification problem (e.g. if there is a human in the room or not) with a small dataset of images from a thermal camera. Originally, those were 7 videos, which I have converted into images of the 224 x 224 pixels. The images are of a still man sitting on a chair, two men sitting on a chair(there are very clear thermal outlines), or an empty chair(mostly a dark background).
I have about 205 images. so I have 76 no human images with label 0 (the empty chair) and 129 human images with label 1 (one man or two men).
I have used a train_test_split() to split these into the test and train datasets proportionally using stratify for an equal representation of labels 0 and 1, and then into the train and validation datasets :
from sklearn.model_selection import train_test_split
X_train_images, X_test_images, y_train_labels, y_test_labels = train_test_split(all_images, all_labels_binary, stratify= all_labels_binary, random_state=0, test_size = .10)
X_train_images, X_val_images, y_train_labels, y_val_labels = train_test_split(X_train_images, y_train_labels,stratify= y_train_labels, test_size=0.30)
My shapes are:
Train: X_train_images=(96, 224, 224, 3), y_train_labels=(96, 1)
Validation: X_val_images=(42, 224, 224, 3), y_val_labels=(42, 1)
Test: X_test_images=(21, 224, 224, 3), y_test_labels=(21, 1)
then I did a similar split to separate into a test and training set.
My model is defined as follows:
cnn_model = tf.keras.Sequential([
Conv2D(32, (5,5), padding = 'same', activation = 'relu', input_shape =(input_shape)),
Conv2D(64, (3,3), padding = 'same', activation = 'relu', input_shape =(input_shape)),
MaxPooling2D(2,2),
Flatten(),
Dense(64, activation = 'relu'),
Dense(1, activation = "sigmoid"),
])
cnn_model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 224, 224, 32) 2432
_________________________________________________________________
conv2d_1 (Conv2D) (None, 224, 224, 64) 18496
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 112, 112, 64) 0
_________________________________________________________________
flatten (Flatten) (None, 802816) 0
_________________________________________________________________
dense (Dense) (None, 64) 51380288
_________________________________________________________________
dense_1 (Dense) (None, 1) 65
=================================================================
Total params: 51,401,281
Trainable params: 51,401,281
Non-trainable params: 0
Now I am having a perfect fit with my validation and test datasets according to my accuracy and loss graphs.
Can someone explain to me why there is a perfect fit (100% accuracy), is it because I have very limited data? what can I do to make my results more meaningful? if I split my videos into more images (say every 3 seconds instead of every 10 seconds), will my results still be the same?
I wonder if try some more models like VGG16 or Resnet in keras, will my results will still be the same with no difference between the models?
Thank you for any advice.