Running CanESM on Google Cloud Platform (Docker)

The new docker container for canesm (note alpha, not functional) can be used to create the same compile/run time environment on any host platform that supports docker/singularity. One such easily accessible platform is Google Cloud Platform. The steps below describe using a Google cloud compute engine VM to compile and run the model (note as of writing, the model does not successfully run yet in this manner).

  • General guidance for running containers on GCP can be found here.

1. Deploy the container on GCE

  • Under Google Cloud Console go to Google Compute Engine

  • I create a compute engine instance (region us-west1), with 16 cores and 60GB of memory.

  • Select “Deploy a container image to this VM instance.”

  • Specify the container image as cccma/canesm-docker:latest

  • Expand “Advanced container options”

  • Select “Allocate a buffer for STDIN” AND “Allocate a pseudo-TTY” (needed for interactive access).

  • Under “Volume mounts” select “Add volume”, then specify Volume type as disk, Mount path as /home/canesm/data, Host path as /home/canesm/data, and Mode as Read/Write.

  • Under Boot disk keep the default container optimized OS, but increase the persistent disk size (I normally take 100 GB, but you can probably get away with 20-30 for testing).

  • create the instance

2. Setup persistent data and access the VM container

  • You can SSH into the VM using something like (requires installing gcloud command line tools):

    gcloud beta compute ssh --zone "us-west1-b" "canesm-docker" --project "canesm"

  • We need data to run CanESM, but every time we close the container, everything is deleted, so we don’t want to download into the container every time. Instead, we we the mounted volume from the persistent disk. So after SSHing into the VM, goto (or make) /home/canesm/data. Go into there and get the configuration data:

    wget ftp://ftp.cccma.ec.gc.ca/pub/CCCMA/nswart/canesm5_piControl_config/*

  • At this stage, you might have to shut down the instance and relaunch it to get the mount properly (uncertain on this, would be a 1 time thing if the mount failed due to /home/canesm/data not existing).

  • Now you can SSH into the actual container itself. To do this you need to know the container name. You can get this on the VM with docker -ps, or you can look in the logs on the cloud console.

  • SSH into the container with something like:

    gcloud beta compute ssh --zone "us-west1-b" "canesm-docker" --project "canesm" --container klt-canesm- docker2-cmab

3. Work in the container

  • You can now go into ~/canesm and try and compile the model (branch gcc-compile and use .gcc Makefiles).

  • You can create a directory at ~/run, link in the data from ~/data, and the exectuables and try to run the model with e.g.

    mpirun -n 1 AGCM : -n 1 CPL : -n 1 nemo.exe

  • At canesm/CCCma_tools/container/tools/ you will find compile and run commands which give examples of how to compile and run the model in the container.