3
$\begingroup$

I currently have collected a dataset of different social media posts for each user with labels assigned to each user. I tried to use LSTM, and BERT for the text classification problem, So for each post, I try to predict the label(for example age). This is not sufficient because you need all of the information contained in the sum of posts to determine the user's age for example. My first thought was to concatenate all of the posts for a single user but since I am currently using BERT which has a maximum sequence length of 512 it wouldn't work. My second idea was to use a text summary and combine them in one vector and hope it doesn't pass the maximum length limit.

Do you have any suggestions for a possible solution? I would assume this problem has been dealt with in the scientific literature and I would be thankful if anyone could point me in the right direction.

$\endgroup$
0
$\begingroup$

You might want to have a look in XLNet.

In XLNet you can input a sentence, then take the state and inject it into a new run if the XLNet with a second input sentence, inject it into a third one with a third sentence and so on.

This, in essence, allows you to process text of virtually unlimited length.

Please see also this.

| improve this answer | |
$\endgroup$
0
$\begingroup$

I was thinking if you could vectorize the entire post using fasttext or gensim. if i understood the problem statement correctly, here is the sample tutorial.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.