******************************************************** 
 Running CanESM on ECCC U2 using imsi
********************************************************

.. contents::
   :local:
   :depth: 3

Overview and resources
######################

This document provides instructions for running the CanESM model on the
ECCC U2 system. This guide is relying on a model configuration system in 
development called ``imsi``, which is in beta testing to replace ``config-canesm`` and ``canesm.cfg``.


Accessing IMSI Commands
-------------------------

To interface with imsi, we provide group cli environments to call imsi functions from.

To activate the CLI environments

.. code-block:: bash

    source /home/scrd102/cccma_libs/imsi/pyenv_3.10_imsi_v0.6.0_2025-04-25/bin/activate

You may also install imsi yourself (see `imsi repo <https://gitlab.com/cccma/imsi>`_).
Read the `imsi documentation <https://imsi.readthedocs.io/en/latest/?version=latest>`_ to get an
overview of how the tools work.

Setup a piControl run
#####################

Navigate to your scratch space (ords or sitestore is good since all code is kept here) and setup a new piControl run as follows [#f1]_:

.. code-block:: bash
    
    imsi setup --repo=https://gitlab.science.gc.ca/CanESM/CanESM5.git --ver=imsi-integration --exp=cmip6-piControl --model=canesm51_p1 --runid=<unique-runid>

where you replace ``--ver`` and ``--exp`` as needed. Be sure to specify a unique runid, that won't clash with other
users, and keep it below 20 characters. Using initials is good. e.g. ``--runid=ncs-hist-tst-01``

If all goes right, you'll eventually see:

.. code-block:: bash

   IMSI setup complete. You can now:

                    cd <runid>

So follow the directions and build the executables (or see `here <#how-to-alter-develop-configurations>`_
to make config modifications prior):

.. code-block:: bash
    
    imsi build

Save the configured restart files:

.. code-block:: bash
    
    imsi save-restarts

Submit the run to the batch queue:

.. code-block:: bash
    
    imsi submit

By default, ``imsi`` will work with ``maestro``, so this will launch the imsi
maestro suite, which has the same structure as the regular maestro suite.
To monitor a run, you can ``export SEQ_EXP_HOME=$(pwd)/sequencer`` and use ``xflow`` as per normal, 
or chain together via:

.. code-block:: bash

   export SEQ_EXP_HOME=$(pwd)/sequencer; xflow

Supported Models/Experiments
############################

In order to see what models/experiments are supported, you can utilize ``imsi
list``. You can either call it from within a setup run directory, or
you can utilize ``--repo /path/to/repo`` to query a specific ondisk repo.
See ``imsi list -h`` for additional details, but simply put:

.. code-block:: bash

    cd /path/to/setup/dir
    imsi list -m # to get models
    imsi list -e # to get experiments

or

.. code-block:: bash

   imsi list --repo /path/to/repo -m
   imsi list --repo /path/to/repo -e

which will give you output about the supported options!

.. note::

   Both models and experiments are known to ``imsi`` as a "selection". If you'd like to change
   the values used after setting up a run, you can utilize ``imsi set`` to reload the configuration
   for the desired model experiment.

   For example - say you wanted to change to a historical experiment, you could:

   .. code-block::

      imsi set -s experiment_name=cmip6-historical

   Or if you want to change to an amip model/experiment

   .. code-block::

      imsi set -s experiment_name=cmip6-amip -s model_name=canam51_p1 # note the second -s

.. warning::

   Not every experiment supports all models contained in the ``imsi`` configurations. To see what models
   are supported for a given experiment, the final source of truth is the ``supported_models``
   `key <https://gitlab.science.gc.ca/CanESM/CanESM5/-/blob/imsi-integration/CONFIG/imsi-config/experiments/cmip6/imsi-exp-config_cmip6-piControl.json#L14>`_ in the config files.

   **Note that if you give an inconsistent set of models and experiments,** ``imsi`` **will tell you the supported options**

How to alter/develop configurations
###################################

Following the directions above, and using supported ``experiment`` and ``model`` combinations, developers and users
can easily setup and launch "canned configurations". However, there will be a time when one wishes
to modify existing configurations to suite their needs or build entirely new experiments, models, post processing
profiles, etc.

To do this, users are directed to the `imsi documentation for common
modifcations <https://imsi.readthedocs.io/en/main/usage.html#modifying-basic-run-parameters>`_, or the more specific links below:

- `changing restarts <https://imsi.readthedocs.io/en/main/usage.html#changing-restarts>`_, ``parent_runid``/``parent_branchtime``
- `altering namelist parameters <https://imsi.readthedocs.io/en/main/usage.html#changing-namelist-parameters>`_
- `changing input files <https://imsi.readthedocs.io/en/main/usage.html#changing-input-files>`_
- `altering diagnostic settings <https://imsi.readthedocs.io/en/main/usage.html#altering-post-processing-settings>`_
- `altering sequencing/resource settings <https://imsi.readthedocs.io/en/main/usage.html#altering-resource-settings>`_
- `adding new experiments and or models <https://imsi.readthedocs.io/en/main/developing_configurations.html#adding-new-models-or-experiments>`_

Setting up a test ensemble
##########################

To aid in the setup of test/development/production ensembles, ``imsi`` has full ensemble support integrated directly into the tool.
Documentation on this useful extension can be found `here <https://imsi.readthedocs.io/en/main/ensemble.html>`_

.. rubric:: Footnotes

.. [#f1] Note, using the `https` address for gitlab is used here as it has no additional requirements. In general using ``--repo=git@gitlab.com:cccma/canesm.git`` is superior, but requires `adding your ssh keys to gitlab <https://docs.gitlab.com/ee/user/ssh.html>`_.