I've developed a deep learning model trained from scratch on fruits and vegetables. However, as the data is limited, I can only cover a few different types of fruits and vegetables with the model alone. However, the accuracy on those categories is very high, 90% training and validation accuracy with around 80% testing accuracy (from a different set of data).
Now I tried to use this trained model on unseen data. So I've removed the softmax layer and only kept the other layers of the model. The idea is to use the feature vector and cosine similarity to compare vegetables.
Let's say bananas were in my original dataset but apples and oranges were not. I use the model to extract features of apples and oranges off of google, then compare the vectors of apples with origins. I would expect the cosine similarity to be very low as they are very different vegetables. However, I get a cosine similarity of 0.9, which is very confusing. To test my sanity, I used a pretrained Mobilenet v2 model and also removed the last layer. Then used the same method to extract the features of apples and oranges. Now they only have a similarity of around 0.5, which is much more reasonable.
I have a few ideas as why this could be the case, perhaps the model could not generalize past the categories given to it during training, or it did not learn similar features that could be translated to apples and oranges so it could not differentiate between the two. However I have no certain explanation. Does anyone have any idea why this is happening, and/or how I can prevent this from happening?