0
$\begingroup$

I have recently read through a lot of documentation and articles about Large Language Models (LLMs), and I have come to the conclusion that 0.7 is, most of the time, the default value for the temperature parameter.

See a few quick reference examples where the default value is either 0.7 or 0.75:

However, I am struggling to find any reference that would explain the rationale for using 0.7.

I understand that lower values of the temperature result in more deterministic outputs and that higher values result in more random outputs.

Nonetheless, why is it more recommended to select temperature=0.7 rather than temperature=0.6 or temperature=0.4 for instance?

In contrast, in "GPT-4 Technical Report", a value of 0.6 is used as the "best-guess" by the authors. See https://arxiv.org/pdf/2303.08774.pdf, p.24.

So my question would boil down to:

- Is it purely empirical or are there either benchmarks, or mathematical equations, which would substantiate the approach of selecting a temperature close to 0.7?

- If it is purely empirical, what were the empirical reasons leading to the adoption of values close to 0.7? (E.g., is it due to the default parameters used in a highly cited paper?, in a highly used library?, etc.)

Thank you

$\endgroup$
2
  • $\begingroup$ Without any proof, I would say that other sites simply copy from the OpenAI example. $\endgroup$
    – noe
    Nov 14, 2023 at 16:00
  • $\begingroup$ Thank you, yes it makes sense and I think you’re right. Then, I guess the answer to my question would be that this is empirical and that the reference leading to the wide adoption of such a default value is simply the OpenAI default setting. I was expecting either studies that would support this choice, or a recommendation of a paper promoting the use of 0.7 in some way. However, I guess I will have to wait for such studies to come out 😄 $\endgroup$
    – jmpion
    Nov 14, 2023 at 16:48

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.