5
$\begingroup$

I am interested in using Google Colab for data modeling. How do I install conda, create an environment and run python in a notebook? I did some searching and found some helpful hints, but had several issues with this. I can only get a partially functional environment so far. I get stuck in running another cell in the same environment. Seems that switching cells resets the environment back to default.

Improve this question
$\endgroup$
5
$\begingroup$

Found a way to get Miniconda working in Google colab. For now, use source activate, not conda activate in the 2nd cell. Still working out the bugs with using conda to activate.

Full Notebook demo here:

https://donaldsrepo.github.io/Notebooks/GoogleColabCondaCreateEnv.html

github with demo notebook:

https://github.com/donaldsrepo/SampleNotebooks/blob/master/CondaCreateEnv/GoogleColabCondaCreateEnv.ipynb

Google uses Python 3.6.9 by default. I created an environment using Python 3.6.10.

Steps to follow:

  1. Install miniconda
  2. Create a new environment
  3. Run some python code to test the environment is being used

Some sample code:

# try to get the bare minimum to get a new conda env working
conda_path = ''
try:
    conda_path = !which conda
finally:
    print('')

if (len(conda_path) == 0):
    print('installing miniconda')
    !wget https://repo.continuum.io/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh && bash Miniconda3-4.5.4-Linux-x86_64.sh -bfp /usr/local
    !conda update conda -y -q
    !source /usr/local/etc/profile.d/conda.sh
    !conda init 
    !conda install -n root _license -y -q
else:
    print('found miniconda')

conda_envs = !conda env list
res = [i for i in conda_envs if 'test36' in i]
if (len(res) == 0):
    print('not found test36 env', len(res))
    !conda create -y -q --name test36 python=3.6
else:
    print('found test36 env', len(res))

Next cell:

%%bash
source activate test36

python
import sys
# maybe only need this the first time we run this notebook
sys.path.append('/usr/local/lib/python3.6/site-packages')

print("Python version")
print(sys.version)

Output:

Python version
3.6.10 |Anaconda, Inc.| (default, May  8 2020, 02:54:21) 
[GCC 7.3.0]

Note that version 3.6.10 is the one installed in my personal conda environment. You can also see your new environment here:

Current Google Colab Notebook

A few things to point out:

  1. If you open a new notebook, your new environment does not exist. Seems to be stored in a transient location, similar to docker.

New Google Colab Notebook

Improve this answer
$\endgroup$
9
  • 1
    $\begingroup$ Hi Donald, I got warning 'source activate myenv' is deprected, and still output shows python points to my default one, not conda's. I think the issue is in !conda init command that seems not being taken into effect. I tried !conda init bash(you need to specify the shell type) but still no effective. Any comment to help out in successfully activating a conda env in colab ? $\endgroup$
    – user305883
    Jun 27 '20 at 13:00
  • 1
    $\begingroup$ You can ignore the warning for now. Did you change source activate myenv to conda activate myenv? If so, this example won't work. It will complain about conda init, etc... For now, can you use source activate? $\endgroup$
    – Donald S
    Jun 27 '20 at 13:56
  • $\begingroup$ Does your screen look like the one on this page? donaldstierman.github.io/Notebooks/CondaCreateEnv.html If not, where does it start to look different? $\endgroup$
    – Donald S
    Jun 27 '20 at 14:17
  • $\begingroup$ Hi Donald, thank you for your quick reply. I re-run now. Screen look the same. Thing is I am creating an environment out of a yml file like this : !conda env create -f enviroment.yml and then run your solution: I confirm it load a python from conda, not sure if from the environment I create. In fact when I run a script, it says cv2 is not installed, but it was successfully installed with the the yml file (no errors). Puzzled. Here you can find full track of what I've done : stackoverflow.com/questions/62610289/… $\endgroup$
    – user305883
    Jun 27 '20 at 16:09
  • $\begingroup$ I think you have to run everything within one cell. The next cell seems to forget some of the information it knew in the previous cell. Also, if you open a new notebook, all the installations will disappear. I suspect Google Colab is doing something similar to running a docker container. It only exists while you are running it and it reverts back to an empty container. They must be doing something special for the notebook file itself, saving it to persistent disk storage. $\endgroup$
    – Donald S
    Jun 27 '20 at 23:05
4
+50
$\begingroup$

Updated Instructions to install Conda on Google Colab Oct 2021

The process is much simpler with condacolab python library

Steps

  1. Import condacolab python library

  2. Install condacolab

    !pip install -q condacolab
import condacolab
condacolab.install()

  3. Post kernel restart, check condacolab installation

    import condacolab
condacolab.check()

Environment

You can update the base environment with an environment file

     !conda env update -n base -f environment.yml

Note : Unfortunately in condacolab only base environment can be used. Avoid creating new environments

Few useful links

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.