0
$\begingroup$

I want to train a network with mBART model in google colab , but I got the message of

RuntimeError: CUDA out of memory. Tried to allocate 886.00 MiB (GPU 0; 15.90 GiB total capacity; 13.32 GiB already allocated; 809.75 MiB free; 14.30 GiB reserved in total by PyTorch)

I subscribed with GPU in colab. I tried to use 128 or 64 for The maximum total input sequence length.

Kindly, What can I do to fix the problem?

$\endgroup$

2 Answers 2

2
$\begingroup$

mBART available in huggingface transformers is not in its base architecture. This means you will not be able to fit this transformer into single GPU offered by Colab.

To deal with GPU OOM problem, you can refer to the PyTorch documentation at https://pytorch.org/docs/stable/checkpoint.html.

Refer to this answer for detailed information.

$\endgroup$
1
  • $\begingroup$ Ok. thanks for your comment $\endgroup$
    – AFB
    Dec 2, 2021 at 6:59
0
$\begingroup$

Adding to the idea of checkpointing proposed by @thanatoz, a few more things you can do in such cases is to use gradient accumulation and mixed-precision training. This will reduce the amount of GPU memory necessary to run the model. I used these techniques to train a custom BERT-large model on long input sequences (512 tokens). Hopefully, it will help you too.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.