1
$\begingroup$

I have a dataset with two input columns as text. Should I use same textvectorizer in both columns or different ones?

I am asking this because. columns a has average length as 500 words, while column b has average length of 50 words.

what and How would you do it?

$\endgroup$
3
  • $\begingroup$ It depends what these columns represent and what is the goal: if they have a really different role with respect to the target class, then it might makes sense to represent them separately, but this has a disadvantage that it generates more features. $\endgroup$
    – Erwan
    Jan 25, 2023 at 13:48
  • $\begingroup$ column a is a paragraph and column b is a question from paragraph. $\endgroup$
    – tikendraw
    Jan 25, 2023 at 15:54
  • $\begingroup$ What are you trying to predict? $\endgroup$
    – Erwan
    Jan 25, 2023 at 16:14

1 Answer 1

0
$\begingroup$

Since you mention that the two texts are related, choose the same text vectorizer.

The average length of each text instance has almost no impact on choosing a text vectorizer.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.