Is there a way of creating a dataloader object, or the equivalent in Keras, where every observation is an image AND some text? I could create two models to do classification but I want to see if I can build it all in one to pull information from both to get the prediction.

Are there any examples of this someone can point me to? Or is it even possible?

The closest I've found is in fast.ai there's this article that combines tabular and text.


I can only guess what you are trying to achieve, but I think this might be helpful even though it is a different approach. The idea is to create two separate models for text and image and merge them into third.

  • $\begingroup$ That's a great example. Pretty simple when you build it that way actually. It's not necessarily building the model directly from both data sources from the start but certainly contains information from both data types. $\endgroup$ – jchaykow Feb 28 '19 at 19:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.