I am new to ML/AI/NLP and am interested in tackling the following problem. I have a database of chat logs from a Discord server. The database contains the following labeled data: Author
, Text
, Text Created Timestamp
, and Channel ID
.
I think what I want to do is fairly complex, but should, in theory, be possible. I want to go through the logs, and group messages that are related to a new label, Conversation ID
. In an ideal world, the model will label each piece of text with a conversation id and I can then run queries against the data to show me a related text that pertains to a specific conversation. among 1 or more individuals. (i say 1 because someone might ask a question and then reply to themselves with an answer. this will be rare)
I can have someone go through the data manually and pull out and label what constitutes a 'conversation.' for some test data. If I was able to build a successful model, I would use the conversations to look for logical groupings of questions asked with follow-up answers. That would be the ultimate goal here is to find questions/answers. I think a logical step would be first to isolate conversations.
The end goal would be to take the discussion from the beginning of time to the start of time and be able to isolate different chunks of text into a logical conversation group.
It would make sense that any conversation happens within the same Channel ID.
I am not worried about cross-channel conversations. Those are rare/unlikely.
My last naive approach for this was to find every question in the corpus and then search for the subsequent N messages following that message from the same channel, then ask an LLM to identify if there was an answer given the question. I want to see if I can improve that process. My approach had a hard limit of messages after a questions where the hard limit was N
, nothing is to say that there wasn't an answer to a question in N+M
messages and I missed the cut-off. The idea is a model won't be limited the number of subsequent messages.
Happy to take advice from seasoned hackers on different approaches to this problem that I can go ahead and experiment with. Names of techniques, tools, models that I can research that apply specifically to this problem would be ideal.