New answers tagged

1 vote

Reducing emails token count preprocessing for Large Email Datasets - Feeding LLMs

Instead of the approaches you mentioned, I suggest a completely different approach: a retrieval-augmented generation (RAG) system. I am doing this because what you described is a typical use case for ...
noe's user avatar
  • 26.6k
0 votes

Recommended way to embed a text thousands of tokens long?

You may use different ways to combine them together, but averaging would be the simplest where you may loose some details and enhance common concepts in the list that you are combining. I would ...
Payam Jome Yazdian's user avatar

Top 50 recent answers are included