I'm trying to find large language models that maps an embedding vector in proximity if they are semantically similar, in Korean. I tried looking at bunch of leaderboard such as MTEB_ko-ko STS, AI Hub benchmark(Korean LLM benchmark), etc... However not all models that I want to compare are within one benchmark therefore hard to compare which one is the best.

So I'm reading about each LLM from its base model, how it is continuously pre-trained to see how its objective function looks like. After shortlisting LLMs I'm planning to create my own dataset to compare all LLMs in shortlists.

As this method seem tidious, wanted to here some ideas on how others will tackle such problem.

  • $\begingroup$ Also does all pre-trained base model such as BERT, Llama get trained to do well on next word prediction task? $\endgroup$
    – haneulkim
    Commented Mar 12 at 6:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.