1
$\begingroup$

I have recently read through a lot of documentation and articles about Large Language Models (LLMs), and I have come to the conclusion that 0.7 is, most of the time, the default value for the temperature parameter.

See a few quick reference examples where the default value is either 0.7 or 0.75:

However, I am struggling to find any reference that would explain the rationale for using 0.7.

I understand that lower values of the temperature result in more deterministic outputs and that higher values result in more random outputs.

Nonetheless, why is it more recommended to select temperature=0.7 rather than temperature=0.6 or temperature=0.4 for instance?

In contrast, in "GPT-4 Technical Report", a value of 0.6 is used as the "best-guess" by the authors. See https://arxiv.org/pdf/2303.08774.pdf, p.24.

So my question would boil down to:

- Is it purely empirical or are there either benchmarks, or mathematical equations, which would substantiate the approach of selecting a temperature close to 0.7?

- If it is purely empirical, what were the empirical reasons leading to the adoption of values close to 0.7? (E.g., is it due to the default parameters used in a highly cited paper?, in a highly used library?, etc.)

Thank you

$\endgroup$
2
  • $\begingroup$ Without any proof, I would say that other sites simply copy from the OpenAI example. $\endgroup$
    – noe
    Commented Nov 14, 2023 at 16:00
  • $\begingroup$ Thank you, yes it makes sense and I think you’re right. Then, I guess the answer to my question would be that this is empirical and that the reference leading to the wide adoption of such a default value is simply the OpenAI default setting. I was expecting either studies that would support this choice, or a recommendation of a paper promoting the use of 0.7 in some way. However, I guess I will have to wait for such studies to come out 😄 $\endgroup$
    – jmpion
    Commented Nov 14, 2023 at 16:48

1 Answer 1

0
$\begingroup$

1.0 is the default,neutral value. What we mean by this is setting to 1 has the (non)effect as if the next token is drawn from the soft-maxed logits without any influence of the

$\endgroup$
1
  • $\begingroup$ As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. $\endgroup$
    – Community Bot
    Commented Apr 15 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.