0
$\begingroup$

I'm a beginner at ML and AI.

Background:

I wanted to try out falcon-7b, the example I'm trying out: https://colab.research.google.com/drive/1BiQiw31DT7-cDp1-0ySXvvhzqomTdI-o?usp=sharing (Falcon-Guanaco example by hugging face team) . But I'm not able to use this due to disk space restrictions. Colab is offering 75GB of disk space and when running the final code block of trainer.train(), the colab's VM runs out of all the disk space it has and fails.

Questions:

  1. Is there anything I can change in that example's code and make it work on a free google colab. I can't get the paid one.
  2. If that's not possible are there any other open source models that I can run on a free google colab that are close to falcon-7b's abilities?
$\endgroup$

1 Answer 1

2
$\begingroup$

Yes there are a couple things you can do to fit the model into google colab's disk.

  1. Reduce the batch size for train and test. This will reduce the GPU memory used for each epoch. Default size set in the colab notebook is 4 but you should try with 2 or even 1.

  2. Increase the save_steps parameter value. This parameter basically saves the model into the disk space after n no of steps. So if the value is 10, then after every 10 steps, the model will be saved in the disk. Increasing this parameter value to more than the default value will result in less no of saves and hence the disk won't get filled. This parameter should be set in tandem with the max_steps parameter. The value of max_steps in the notebook is 500. So it is best if you set the value of save_steps to 50 or maybe even 100. Keep in mind this does not affect the model performance at all. This is just used to save the trained model after every few steps so in case your training stops or runtime gets disconnected, you still have the last trained model.

  3. Add the parameter save_total_limit. This will limit the number of models to be saved. So if you set this value as 3 then the last 3 trained models will be saved into the disk. You don't need this value to be more than 1 as you only need the last trained checkpoint of the model. This is the most probable cause why your model is not fitting into the colab's disk.

Using the above 3 parameters,you will definitely be able to train the model! Let me know if you were able to fine tune it.

Cheers!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.