Let's say, one wants to detect, whether a picture of a fixed size contains a cat or not. But as a dataset, you have 10000 pictures of cats, and 30000 pictures which don't contain a cat, but are very similar to each other. For example, let's assume, the 30000 pictures in the "not cat" class contain only pictures of one or two kinds of spiders.
When training a CNN, you will find that you achieve a high score on the test set (here high score = almost fully diagonal confusion matrix) but when you want to use the CNN in the real world, you find that almost everything gets classified as a cat.
Why does the network generalize badly in this case? Even if the dataset doesn't represent the kind of data, the CNN would see in the real world, shouldn't it be easy for the CNN to say "I have seen 10000 examples of cats, therefore anything which doesn't look like a cat is not a cat" ?
How would one deal with this problem (besides gathering more data)?