Hybrid classification neural network

Question

I have product data and I need to classify products to categories (for example Lenovo laptop to Laptops category, etc.), each product has properties such as:

1. description
2. list with image URLs (typically 4 photos)
3. product-specific properties (watches have a mechanism type attribute, etc.)
4. manufacturer
5. category ID

Category ID is my target variable, do you know some resources (articles/books) where someone did something similar? I heard about the transfer learning (answer to this question Hybrid Convolutional and Conventional Neural Networks, is it a good approach?). My biggest problem is that I don't know how to connect CNN for images similarity and conventional neural network. Thanks for the help.

Answer 1

Basically, you need to create a whole system which contains multiple ML algorithms. We can go feature-wise and see how the system could work.

1. Description: This will consist of text which could be easily vectorized using Doc2Vec. This vector will act as a feature for the final model.
2. Product images: You will need to create a model which make some sort of classification. Once, the model is trained to remove the last layer ( mostly the Softmax layer ). This will fetch us the encoded representation of the image. You may use an AutoEncoder for this.
3. Manufacturer and other properties: I think you may need to omit this feature as they will be product-specific and create complexities while training.

Once, we have all the above models trained, we can feed the features ( generated from these models ) to a neural network, which will produce the final result.