CanESM Dependencies
===================

Arguably the most important part of the porting effort is to get ``CanESM`` to
run on the desired platform, which requires specific software libraries/stacks
to be built and available.

.. contents::
   :local:
   :depth: 2

Compilers and Supporting Utilities
----------------------------------

Given ``CanESM`` is a compiled model, `largely` made up of ``Fortran`` codes but with a small
(but increasing) amount of ``C`` and ``C++`` codes, it requires a compilation system to translate
the source files into a machine executable.

.. note::

   At the time of writing this, ``CanESM`` has mainly been tested with the ``intel``
   compilation toolchain, so ``ifort``, ``icc``, and ``icpc`` (and eventually the ``LLVM``
   equivalents, ``ifx``, ``icx``, ``icpx``). Other compiler tool-chains, like the ``GNU`` version,
   `should` work as well, but extra testing is required. In the future, it is hoped that
   ``CanESM`` will be more robustly tested with other compilers.

.. note:: 

   Depending on how the ``MPI`` implementation is setup for your platform, its possible there will
   be ``mpi`` wrappers of these compilers - for example ``mpiifort``.

Fortran Compiler
~~~~~~~~~~~~~~~~

The exact version of the compiler can vary, as such this doc reports the various versions
that have been tested extensively

**on niagara**:

.. code-block::

   ifort >= 19.1.1.217 20200306 # loaded from module intel/2020.1.217

**on eccc-u2 platforms**:

.. code-block::

   ifort >= 2021.10.0 20230609  # note that the versioning behaviour of ifort has recently changed

C/C++ Compiler
~~~~~~~~~~~~~~

The exact version of the compiler can vary, as such this doc reports the various versions
that have been tested extensively

**on niagara**:

.. code-block::

   icc  >= 19.1.1.217 20200306
   icpc >= 19.1.1.217 20200306

**on eccc-u2 platforms**:

.. code-block::

   icc  >= 2021.10.0 20230609
   icpc >= 2021.10.0 20230609

Gnu-Make
~~~~~~~~

Multiple components with in the ``CanESM`` eco-system rely on ``Makefiles``
to define the build dependencies, as such new platforms must have a version
of ``Gnu-Make`` installed. Specifically:

.. code-block::

   gnu-make >= 4.2.1 # note that version can be checked with make --version

It is possible earlier versions would work as well, but testing is required.

Cmake
~~~~~

.. note::

   **This is only required for CanESM versions >= 6.0** - specifically it is used within the
   ``GEM`` build systems. So this can be ignored for model versions using the spectral dynamical core.

For specific versions of ``CanESM``, the development team has started using ``CMake``
to generate ``Makefiles``, and thus a version of ``CMake`` must be available to 
build these later versions of ``CanESM``. Specifically:

.. code-block::

   cmake >= 3.16

Again, earlier versions might work, but testing is required.

Perl 5
~~~~~~

Underneath the hood, the build utilities utilized by the ocean and coupler rely
on ``Perl5`` in some fashion. Generally, any version of ``Perl5`` should
suffice, but its possible your platform might have minimal ``perl`` module
support and you need to install extra perl modules (for example, on ``Niagara``
we needed to install ``URI`` manually) -  if this is the case, simply point to
the installation directory with ``PERL5LIB`` and include that consideration
in the definition of the machine environment files.

MPI
---

The main parallelization method that ``CanESM`` utilizes is "distributed memory parallelization", which
is achieved via ``MPI`` or the "Message Passing Interface". There are various implementations of this,
all of which should work:

* `OpenMPI <https://www.open-mpi.org/>`_
* `IntelMPI <https://www.intel.com/content/www/us/en/developer/tools/oneapi/mpi-library.html>`_
* `MPICH <https://www.mpich.org/>`_

Over the recent years of ``CanESM`` development, all of these `have` been used with ``CanESM``.
However, for recent platforms the versions tested have been:

**on niagara**:

.. code-block::

   Open MPI >= 4.0.3

**on eccc-u2**:

.. code-block::

   Intel MPI >= 2021.10 # comes with the intel compiler toolchain noted above

.. note::

   Versions and Implementations have been tested via

   .. code-block::

      mpirun --version

OpenMP
------

For certain implementations of the atmosphere component of ``CanESM``, `OpenMP
<https://www.openmp.org/>`_ is utilized for shared memory parallalization.
Fortunately, ``OpenMP`` support is typically tied to the compiler being used,
so provided the compiler your platform will use has ``OpenMP`` support, nothing
extra is required.

Netcdf Fortran
--------------

While the atmosphere component of ``CanESM`` utilizes prorietary, binary data
formats, both the ocean and coupler componenets utilize the community standard,
`NetCDF <https://www.unidata.ucar.edu/software/netcdf/>`_ file format. As such,
``CanESM`` requires a version of ``NetCDF-Fortran`` to be compiled and available.

.. warning::

   ``NetCDF-Fortran`` should be built using the same compiler that will be used
   for the model. In some cases, it will work if this isn't true, but it can lead
   to `less-than-obvious` downstream problems.

Similar to other requirements, the exact version may change, but for recent
versions of ``CanESM`` these are the tested versions on the various platforms:

**on niagara**:

.. code-block::

   netCDF-Fortran >= 4.5.2

**on eccc-u2**:

.. code-block::

   netCDF-Fortran >= 4.4.5

.. note::

   versions were checked with

   .. code-block::

      nf-config --version

ESMF
----

To handle the regridding operations between model components, the coupler
utilizes `ESMF <https://earthsystemmodeling.org/>`_, or the "Earth System
Modeling Framework".

On both the ``niagara`` and ``eccc-u2`` platforms, the tested versions have been

.. code-block::

   ESMF >= 8.0.1

However it is recommended to attempt to jump to the latest version of ``ESMF``
for new platforms.

.. warning::

   ``ESMF`` should be built using the same compiler that will be used for the model.
   In some cases, it will work if this isn't true, but it can lead to `less-than-obvious`
   downstream problems.

.. warning::

   For the version used on the ``eccc-u2`` platform (8.1.1), there were initially bit
   reproducibility problems that stemmed from the way ``ESMF`` was built. See
   `here <https://github.com/uturuncoglu/RegESM/blob/master/docs/07_Known_Issues.md#74-bit-to-bit-reproducibility>`_
   for details (specifically commit: ``5e9b0ea7e3dc47f992d4c6362b83b02e0ee832ba``). 
   To address this on the ECCC platforms, the
   ``build_config/Linux.intel.default/build_rules.mk`` file was modified to have

   .. code-block::

      ESMF_CXXCOMPILEOPTS += -fp-model precise
      ESMF_CXXLINKOPTS += -fp-model precise
      ESMF_F90COMPILEOPTS += -fp-model precise
      ESMF_F90LINKOPTS += -fp-model precise

FFTW
----

.. note::

   **This is only required for CanESM versions >= 6.0** - specifically it is used within the
   ``GEM`` dynamical core. So this can be ignored for model versions using the spectral dynamical core.

For later versions of the atmosphere component (used in ``CanESM 6``),
`FFTW <https://www.fftw.org/>`_ is used for fast fourier transforms. Specifically,
recent testing has used:

.. code-block::

   fftw >= 3.3.10

.. warning::

   Like ``ESMF`` and ``NetCDF-Fortran``, this library should be built with the same
   compiler that will be used for the model.