I am interested in using Google Colab for data modeling. How do I install conda, create an environment and run python in a notebook? I did some searching and found some helpful hints, but had several issues with this. I can only get a partially functional environment so far. I get stuck in running another cell in the same environment. Seems that switching cells resets the environment back to default.
2 Answers
Updated Instructions to install Conda on Google Colab Oct 2021
The process is much simpler with condacolab
python library
Steps
Import
condacolab
python libraryInstall
condacolab
!pip install -q condacolab import condacolab condacolab.install()
Post kernel restart, check condacolab installation
import condacolab condacolab.check()
Environment
You can update the base environment with an environment file
!conda env update -n base -f environment.yml
Note : Unfortunately in condacolab only base environment can be used. Avoid creating new environments
Few useful links
- Help documentation
- Demo notebook in Google colab
- Additional link
- Alternatively, you can access google colab in conda as well
Found a way to get Miniconda working in Google colab. For now, use source activate, not conda activate in the 2nd cell. Still working out the bugs with using conda to activate.
Full Notebook demo here:
https://donaldsrepo.github.io/Notebooks/GoogleColabCondaCreateEnv.html
github with demo notebook:
Google uses Python 3.6.9 by default. I created an environment using Python 3.6.10.
Steps to follow:
- Install miniconda
- Create a new environment
- Run some python code to test the environment is being used
Some sample code:
# try to get the bare minimum to get a new conda env working
conda_path = ''
try:
conda_path = !which conda
finally:
print('')
if (len(conda_path) == 0):
print('installing miniconda')
!wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh && bash Miniconda3-4.5.4-Linux-x86_64.sh -bfp /usr/local
!conda update conda -y -q
!source /usr/local/etc/profile.d/conda.sh
!conda init
!conda install -n root _license -y -q
else:
print('found miniconda')
conda_envs = !conda env list
res = [i for i in conda_envs if 'test36' in i]
if (len(res) == 0):
print('not found test36 env', len(res))
!conda create -y -q --name test36 python=3.6
else:
print('found test36 env', len(res))
Next cell:
%%bash
source activate test36
python
import sys
# maybe only need this the first time we run this notebook
sys.path.append('/usr/local/lib/python3.6/site-packages')
print("Python version")
print(sys.version)
Output:
Python version
3.6.10 |Anaconda, Inc.| (default, May 8 2020, 02:54:21)
[GCC 7.3.0]
Note that version 3.6.10 is the one installed in my personal conda environment. You can also see your new environment here:
A few things to point out:
- If you open a new notebook, your new environment does not exist. Seems to be stored in a transient location, similar to docker.
-
1$\begingroup$ Hi Donald, I got warning 'source activate myenv' is deprected, and still output shows python points to my default one, not conda's. I think the issue is in
!conda init
command that seems not being taken into effect. I tried!conda init bash
(you need to specify the shell type) but still no effective. Any comment to help out in successfully activating a conda env in colab ? $\endgroup$ Jun 27, 2020 at 13:00 -
1$\begingroup$ You can ignore the warning for now. Did you change
source activate myenv
toconda activate myenv
? If so, this example won't work. It will complain about conda init, etc... For now, can you use source activate? $\endgroup$– Donald SJun 27, 2020 at 13:56 -
$\begingroup$ Does your screen look like the one on this page? donaldstierman.github.io/Notebooks/CondaCreateEnv.html If not, where does it start to look different? $\endgroup$– Donald SJun 27, 2020 at 14:17
-
$\begingroup$ Hi Donald, thank you for your quick reply. I re-run now. Screen look the same. Thing is I am creating an environment out of a yml file like this :
!conda env create -f enviroment.yml
and then run your solution: I confirm it load a python from conda, not sure if from the environment I create. In fact when I run a script, it says cv2 is not installed, but it was successfully installed with the the yml file (no errors). Puzzled. Here you can find full track of what I've done : stackoverflow.com/questions/62610289/… $\endgroup$ Jun 27, 2020 at 16:09 -
$\begingroup$ I think you have to run everything within one cell. The next cell seems to forget some of the information it knew in the previous cell. Also, if you open a new notebook, all the installations will disappear. I suspect Google Colab is doing something similar to running a docker container. It only exists while you are running it and it reverts back to an empty container. They must be doing something special for the notebook file itself, saving it to persistent disk storage. $\endgroup$– Donald SJun 27, 2020 at 23:05