Once for the task of image captioning I've read that, the features extracted from image and text by deep networks are from two different worlds and got different distribution. My question is how is the distribution in two of them and how are they different?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Suppose you trained two identical neural nets on different datasets. Network A is trained using a dataset of cat pictures. Network B is trained using a dataset of traffic sign images. Because the two networks are identical, they will obviously produce a feature map in the same space, right? But the distribution of features in that space will be different for the two networks, because they were trained on different datasets, and you need different feature extractors to recognize cats vs. traffic signs.
This is analogous to what you have read about the text/image features. Suppose we train a network to embed image data in some N-dimensional space, and then we train another network to embed text data in the same N-dimensional space. Although the resulting feature vectors are in the same space, they are almost certain to have different distributions, because they were trained using different datasets.
Unfortunately, we cannot give a general answer about how the distributions are shaped and what exactly the differences are. These details will vary from case to case. Although we may not know exactly how the distributions are different until we have them in hand, we can be confident that they are in fact going to have substantial differences.
