The CanESM Development Cycle is the effort that integrates
improvements in CanESM to form new release versions of the model for scientific
applications. The development cycle coordinates work across multiple development
teams, and includes changes ranging from technical fixes to incorproration of
new model components. The goals, policies and processes governing the
development cycle are laid out in this document.
At the highest level, individual features or improvements are continuously
implemented by developers, in consultation with the CCCma team working in a
specific area (atmosphere, ocean or land surface). The CanESM working group
periodically assembles new versions of the coupled model, with new features actived on
advice of the working team leads. After testing and scientific verification, a new
tagged version of CanESM is released for further development and scientific applications.
This process is further detailed below.
The overarching goals of this coordinating effort are:
To create a series of functional versions of CanESM, that represent
incremental improvements on each other, that can be used for ongoing
scientific applications, as well as serving as a basis for model development.
To ensure that models used in major scientific applications are as robust
and thoroughly vetted as possible. This favours frequent, incrementally
developed versions which allow time for vetting, and precludes last minute
combinations.
To improve development efficiency by using modern tools and software engineering
principles.
The development cycle strives to seek a balance between making the latest
improvements availble, while maintaining stability for developers and avoiding code divergence
or stagnation.
An official model version is one that has been nominally validated and released by the CanESM
working group for scientific applications and futher development. An official model version is
also referred to as a tagged release, because each official version is associated with a tag
(e.g. v5.0.9) in the CanESM repositories. Official versions are easily locatable on the
branch named canesm (See branches).
Version numbering for CanESM follows the standard [1] of M.m.p, where:
M is a major version number, indicating substantial changes in the model and subcomponents.
m is a minor version number, generally reflecting incremental scientific and technical changes.
p is a patch version number, generally reflecting technical improvements and scientific bugfixes.
Patch version numbers are generally minted to note bit pattern changes or other notable milestones between minor releases.
There is no fixed release schedule, however, minor release versions might be anticipated on a timescale of
roughly 1 year, with patch release versions being minted every several months.
The process for creating and releasing more significant changes found under minor versions is as follows:
The starting point is generally a previously tagged release, and integration of changes is done on the branch develop_canesm.
A strict testing regime is maintained to ensure all changes are well understood (see below).
Features and infrastructure are developed as described below, at the leisure of developers, and merged
into develop_canesm as they become available, maintaining the bit-pattern of the model so far as
possible (i.e. any new physics has a switch which is deactivated).
At the beginning of a development cycle, the CanESM working group will mint a new release branch (e.g. v5.2_release).
The release branch is used to activate new physics (via switches) previously merged to develop_canesm (but switch off there).
New features (branches) can also be merged to the release branch, when switching is not possible, and follow the testing regime.
For example, the inclusion of a major new component model or wholesale replacement of a major piece of infrastructure typically
is not “switched”, but if it can be that is desirable. New physics and infrastructure will be activated
on the release branch based on the guidance of the sub-component working group leads in conjuction with the CanESM development team.
Release branches will be kept up to date with develop_canesm for the duration of their existance.
Over the course of the development cycle, new features will accumulate on the release branch. Tags on the release branch
will be used to indicate milestone progress. Tags will proceed from alpha, beta through to release candidates. For example,
v5.2.a1, v5.2.a2, v5.2.b, v5.2.rc1. Alpha releases will indicate raw combinations, beta releases will reflect
harmonized combinations of code, and release candidates will indicate a near final version of the official version. Closing
release candidates is known as “freezing” the model.
Release candidates on the release branch will be used by the CanESM group to create a new tuned version of the coupled model,
in conjuction with evaluation efforts from the Analysis for Development team, and general feedback from sub-component
working groups. This process is also called finalization.
Prior to each final minor release, relevant DECK+ documenting simulations will be produced for internal reference, and
possible publication via appropriate channels (such as ESGF or other open data platforms).
Once a validated release candiate is accepted, the CanESM group will mint the official new minor verion of
CanESM. The release branch will be merged back to the develop_canesm and canesm branches. The new release will then
be the official model version. The temperory release branch will be deleted.
The process above will naturally create a continuous progression of tuned models, that can be deployed for science
applications.
A solid testing regime is a cornerstone of a reliable code in which mutliple developers contribute in parallel. All code integrated
into develop_canesm and release branches occurs through Merge Requests, that include in them a reporting of the testing undertaken.
All MRs are expected to preserve the functionality of the target branch, and in general to also preserve the bit pattern wherever possible.
A series of standard one year simulations comprising the “DECK+” experiments are required on all MRs back to offical branches (including release*). These verify functionality and bit
pattern. Bit identical code which preserves all tested functionality maybe merged back immediately, following approval of a code review by
relevant experts.
In cases where a bit change is expected and unavoidable, even at low optimization levels, it is required that at minimum >100 year
piControl simulation is conducted to evaluate the climate. Based on the changes and results of this test, additional and longer simulations
might be required.
For new physics feature development, more comprehensive tests (including of climate change response) will
ultimately be needed before they are activated. The relevant tests would be determined by the relevant working group (see below).
Before any significant development of CanESM is undertaken, the proposed changes should be listed in an issue along with their rationale
and physical basis, and discussed with the relevant working group and ESM teams. This helps to ensure proposed development aligns with the
priorities and directions of CanESM from the start. Once development is complete, comprehensive climate tests are needed to assess any
physical or significant technical changes. WGs will undertake an internal process to evaluate changes made in their areas, and based on
this will recommend changes for inclusion (or not) in the next minor release of CanESM. There is no guarantee whatsoever that changes/features
will be accepted into the model, but comprehensive consulatation will help to create the best chance of inclusion.
this persistent branch is kept completely stable between tagged versions of the
model. Prior to updating this branch with a new tag, it is confirmed that a
comprehensive set of experiments/configurations are operational using the
version in question.
develop_canesm:
this persistent branch is considered “bleeding-edge” and is the version that the
majority of development is based on. This branch is updated regularly
with technical fixes/updates, along with deactivated physical changes.
Due to this continuous integration, this branch may not always be fully
functional - while effort is made to keep it so, users are encouraged to
use canesm or a tagged version of the model for their scientific
applications. The bit pattern of this branch will change in time, but
such changes should always be demarked by at minimum a patch version increment.
vM.m_release
this is a tempory branch, used to activate new physics and include major new
components for testing. It is intended to test climate changing modifications.
The release branch is created from the previous minor release,
and kept up to date with develop_canesm during its life. After the official
release tag is minted, the release branch is merged back to canesm and
develop_canesm, and is then deleted.
