I ran my job on a computing cluster: first with 1 node / 4 cores, then with 2 nodes / 32 cores. But the training time is pretty much exactly the same for both of them! 67 seconds per step.

I am trying to fine-tune GPT-2 for my text dataset (chat logs).

What can I do to get a performance increase corresponding to the increase in processing power with CPUs?


1 Answer 1


Have you done any analysis as to what steps in your procedure consume most time? For example, it may be that GPU operations are responsible for most of the time. Then, of course, adding CPU cores will not give any benefit. And have you looked at CPU statistics, i.e. is the full capacity used?

  • $\begingroup$ Why wouldn't adding CPU cores give any benefit? Even if a GPU performs the operations faster than a single CPU, can't it distribute the operations across multiple CPUs to improve performance too? The "htop" process monitor shows ~90-something% usage when using the model, iirc. $\endgroup$ Commented Nov 14, 2019 at 3:31
  • $\begingroup$ It really depends on the computations done. If CPU operations are done in milliseconds and GPU operations run for seconds, then speeding up the computational time by a few milliseconds will not show any benefit. (Especially since the CPU is also used by the operating system.) Can you please clarify what operations you do? $\endgroup$ Commented Nov 18, 2019 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.