0
$\begingroup$

So I have three types of data (in title) and am wondering how I can combine the data. The target is numeric (price).

My idea is to perform feature extraction on both the images and text, which would result in a 1 dim row vector of size n. So this would produce n features. After this, combine all the data and normalize. The model would be trained on this combined dataset.

I have really only worked with one data type or the other, so I am unsure if this is the right approach. I haven't found many relevant resources for this type of problem, but let me know if I am missing something.

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
0
$\begingroup$

Based on the assumption that you can correctly assign a text, image and numeric features to each instance of your dataset, I would try as a first approximation:

  • numeric features: as is (applying usual preprocessing on numeric features if necessary)
  • text: treat, at the beginning, the texts in your dataset as an independent set, build your bag of words, and find the most frequent terms (uni or bi-grams) so that you can add the frequency of such terms as new features (adding these ones to the numeric features of point 1)
  • images: if you have in advance an idea of which kind of objects you will have in those images, you can build an image classifier (or even an object recognition model), so the output of this model would be the value of the additional features (categorical values) of which type of image/objects your image is (here an introductory source in case it's needed).

In the end, you would end up with something like:

id numeric_1 numeric_2 numeric_3 term_1_freq term_2_freq term_3_freq term_4_freq car cat beach dog
1 3 0.4 20 0 0.3 0 0.6 0 0 1 0
2 3 0.2 25 1 0.2 0.05 0.7 0 0 0 1
3 3 0.4 10 0 0.2 0 0.6 0 1 1 0
Improve this answer
$\endgroup$
2
  • $\begingroup$ So just to be clear, what you are suggesting is that treat each type of data separately (and by treat you mean convert them into numeric values by BOW and image classification) and then combine all three data into one and then perform regression? $\endgroup$
    – spectre
    Oct 25, 2021 at 11:43
  • $\begingroup$ indeed, that is one possible suggestion to begin with, you could try other approaches later on $\endgroup$
    – German C M
    Oct 25, 2021 at 12:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.