I am trying to build a classifier for a specific card dataset let's say cards or no cards. I am using Mobilenet trained on Imagenet dataset as my classifier and further training it on my dataset. I am able to train it, and it's performance is quite good on the dataset. Let's say my card has for different regions of interest as shown below:-
It is able to perfectly recognize the above passed image as card.
I want the classifier to learn the exact occurrences of features of cards as in First Image so that if I alter them it should be able to reject that card (like in second image).
Is there any way to achieve this?