1
$\begingroup$

Hello i am having a classification between two classes A and B and i have trained CNN model. I have high accuracy on all three set of data i.e training (98.7%) validation (99.3%) and test(98%) but still can not predict on real data could any please help to sort out the issue.
p.s i have balance data.

train_augmented = ImageDataGenerator(rescale=1./255,
                                   rotation_range = 40,
                                   horizontal_flip = True,
                                   
                                   zoom_range = 0.2
                                   )
test_augmented = ImageDataGenerator(rescale=1./255)
validation_augmented = ImageDataGenerator(rescale=1./255)

training_set = train_augmented.flow_from_directory(train,
                                                    target_size=(150,150),
                                                    batch_size = 32,
                                                    class_mode = 'binary',
                                                    
                                                    color_mode="rgb",
                                                    shuffle=True
                                                    )
test_set = test_augmented.flow_from_directory(test,
                                                    target_size=(150,150),
                                                    batch_size = 32,
                                                    class_mode = 'binary',
                                                    color_mode="rgb",
                                                    shuffle=False
                                                    )
validation_set = validation_augmented.flow_from_directory(validation,
                                                    target_size=(150,150),
                                                    batch_size = 32,
                                                    class_mode = 'binary',
                                                    color_mode="rgb",
                                                    shuffle=False
                                                    )
model = Sequential()
model.add(Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3)))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.5))

model.add(Conv2D(64,(3,3),activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.5))

model.add(Flatten())
model.add(Dense(256,activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1,activation='sigmoid'))
model.summary()
model.compile(Adam(learning_rate=0.001),loss='binary_crossentropy',metrics=['accuracy'])
history = model.fit_generator(training_set,
                    epochs = 30,
                    validation_data = validation_set)
$\endgroup$
1
  • 5
    $\begingroup$ With this little information it's going to be difficult to diagnose the problem. We don't know what your data is, how you process it, how you train and validate your model. As a guess, if your model performs well during training and validation but not on real-world data, your training data does not match your real-world data or you did a mistake during training which lead to your model overperforming on the validation data but not generalizing correctly. $\endgroup$ May 22, 2023 at 13:44

2 Answers 2

0
$\begingroup$

It means that your training/validation/test data is not representative of the real world.

You should get fresh new data from the real world, and use it instead.

Of course I trust that your training/validation/test directories do not contain duplicates.

$\endgroup$
0
$\begingroup$

First of all, your dataset is it balanced ? If not, accuracy is a bad metric to use because it will be strongly influenced by the most represented class. You can use F&-Score which will be better.

Are you sure that your test set is different than your training set ?

Otherwise, like @nicolas-raoul suggest try to be sure that your training / test set are the same.

Good luck with your investigation !

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.