I am interested in using Google Colab for data modeling. How do I install conda, create an environment and run python in a notebook? I did some searching and found some helpful hints, but had several issues with this. I can only get a partially functional environment so far. I get stuck in running another cell in the same environment. Seems that switching cells resets the environment back to default.
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
Found a way to get Miniconda working in Google colab. For now, use source activate, not conda activate in the 2nd cell. Still working out the bugs with using conda to activate.
Full Notebook demo here:
https://donaldsrepo.github.io/Notebooks/GoogleColabCondaCreateEnv.html
github with demo notebook:
Google uses Python 3.6.9 by default. I created an environment using Python 3.6.10.
Steps to follow:
- Install miniconda
- Create a new environment
- Run some python code to test the environment is being used
Some sample code:
# try to get the bare minimum to get a new conda env working
conda_path = ''
try:
conda_path = !which conda
finally:
print('')
if (len(conda_path) == 0):
print('installing miniconda')
!wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh && bash Miniconda3-4.5.4-Linux-x86_64.sh -bfp /usr/local
!conda update conda -y -q
!source /usr/local/etc/profile.d/conda.sh
!conda init
!conda install -n root _license -y -q
else:
print('found miniconda')
conda_envs = !conda env list
res = [i for i in conda_envs if 'test36' in i]
if (len(res) == 0):
print('not found test36 env', len(res))
!conda create -y -q --name test36 python=3.6
else:
print('found test36 env', len(res))
Next cell:
%%bash
source activate test36
python
import sys
# maybe only need this the first time we run this notebook
sys.path.append('/usr/local/lib/python3.6/site-packages')
print("Python version")
print(sys.version)
Output:
Python version
3.6.10 |Anaconda, Inc.| (default, May 8 2020, 02:54:21)
[GCC 7.3.0]
Note that version 3.6.10 is the one installed in my personal conda environment. You can also see your new environment here:
A few things to point out:
- If you open a new notebook, your new environment does not exist. Seems to be stored in a transient location, similar to docker.
-
$\begingroup$ Hi Donald, I got warning 'source activate myenv' is deprected, and still output shows python points to my default one, not conda's. I think the issue is in
!conda initcommand that seems not being taken into effect. I tried!conda init bash(you need to specify the shell type) but still no effective. Any comment to help out in successfully activating a conda env in colab ? $\endgroup$ – user305883 Jun 27 at 13:00 -
$\begingroup$ You can ignore the warning for now. Did you change
source activate myenvtoconda activate myenv? If so, this example won't work. It will complain about conda init, etc... For now, can you use source activate? $\endgroup$ – Donald S Jun 27 at 13:56 -
$\begingroup$ Does your screen look like the one on this page? donaldstierman.github.io/Notebooks/CondaCreateEnv.html If not, where does it start to look different? $\endgroup$ – Donald S Jun 27 at 14:17
-
$\begingroup$ Hi Donald, thank you for your quick reply. I re-run now. Screen look the same. Thing is I am creating an environment out of a yml file like this :
!conda env create -f enviroment.ymland then run your solution: I confirm it load a python from conda, not sure if from the environment I create. In fact when I run a script, it says cv2 is not installed, but it was successfully installed with the the yml file (no errors). Puzzled. Here you can find full track of what I've done : stackoverflow.com/questions/62610289/… $\endgroup$ – user305883 Jun 27 at 16:09 -
$\begingroup$ I think you have to run everything within one cell. The next cell seems to forget some of the information it knew in the previous cell. Also, if you open a new notebook, all the installations will disappear. I suspect Google Colab is doing something similar to running a docker container. It only exists while you are running it and it reverts back to an empty container. They must be doing something special for the notebook file itself, saving it to persistent disk storage. $\endgroup$ – Donald S Jun 27 at 23:05
