Forking CanESM ============== For generic information on forking, see :ref:`Forking workflow references `. Creating Your Forks ------------------- As part of a forking workflow, the first thing you need to do is create your forks! whether you intend to modify them or not, forks must be taken for each and every submodule. To do so, follow the steps below. .. note:: You only need to fork the repositories once! Thereafter you can interact with you forks as needed. 1. Navigate to the `CanESM Repository `_ and fork the repository into your namespace by clicking the "Fork" button on the project dashboard .. image:: /images/ForkButton.png :align: center :alt: Fork Button which will bring you to a **Fork Project** menu. From here, select your own personal namespace, i.e. .. image:: /images/NameSpaceButton.png :align: center :width: 150 :alt: Name Space Button At this stage, you should then be on your own personal version of the CanESM super repository. In the top left you should see something similar to, .. image:: /images/NameSpaceEx1.png :align: center :width: 200 :alt: Name Space Example with your name instead. Additionally, in the ``ssh`` address section you should see: .. code-block:: shell git@gitlab.science.gc.ca:user123/CanESM5.git where ``user123`` is replaced with your account ID. 2. Fork the submodules by navigating to the project pages: - `CCCma_tools `_ - `CanAM `_ - `CanCPL `_ - `CanDIAG `_ - `CanNEMO `_ - `PAM `_ .. WARNING:: If you fail to fork all of the submodules, this will be fatal for usability. If you try to navigate to a submodule from within your fork and you get a "Not Found" error, you haven't forked the repository. 3. (*recommended*) Disable the build feature for your forks. Unless you want to setup automatic builds on your forks, which requires some knowledge of gitlab runners, it is recommended that developers disable this feature on their forks. **If developers do not do this, their forks will ATTEMPT to launch continuous integration pipelines in the same way the central repo does**. However, because a gitlab runner hasn't been setup for their fork, **the pipelines will ALWAYS fail**, at *no fault of the developer*, producing un-needed notifications for the developer. To disable this on your fork, navigate to the project settings menu, which is the small gear upper right-hand corner of the project dashboard, i.e. .. image:: /images/ForkDashboard.png :align: center :alt: Project Dashboard From the drop down menu, select "Edit Project" .. image:: /images/ForkDashboard_Settings.png :align: center :alt: Edit Project Button and you should now see a "Project Settings" interface. Scroll down to the "Feature Visibility" sub-menu and take note of the "Builds" setting: .. image:: /images/Feature_Visibility.png :align: center :alt: Feature Visibilty Default Open the drop down menu and set this to "Disabled: .. image:: /images/Feature_Visibility_Builds_Disabled.png :align: center :alt: Feature Visibility With Builds Disabled then scroll down and select "Save Changes", and with that the builds feature will be turned off! The last step is to repeat this for the submodules! Interacting With the Main Repo (and other remotes) -------------------------------------------------- Due to using a forked workflow model, ``CanESM`` contributors need to be comfortable interacting with multiple remotes. This section provides specific examples and details about some of the common operations developers will need to do. It should be noted that in the below sections, while it is assumed that the on-disk repos have been cloned from user forks on the internal ECCC ``gitlab`` server, i.e. .. code-block:: text git@gitlab.science.gc.ca:user123/CanESM5.git and that the "other" remote we use as an example is the central repository at .. code-block:: text git@gitlab.science.gc.ca:CanESM/CanESM5.git these directions can easily be applied to other, more generic remotes Additionally, the instructions below utilize many of the internally made `s-scripts`. For information on these scripts, and how to accessed pre-installed versions on the ECCC machines, interested readers are directed :ref:`here `. Adding a new remote to your on-disk repo """""""""""""""""""""""""""""""""""""""" By default, when you clone down *any* repository ``git`` sets up the remote url/name for you, where the default remote name is ``origin`` and it points to the url you cloned from. For example if we clone down the ``CanESM`` repo via .. code-block:: text >> git clone --recursive git@gitlab.science.gc.ca:user123/CanESM5.git then the default remote setup will be .. code-block:: text >> git remote -v origin git@gitlab.science.gc.ca:user123/CanESM5.git (fetch) origin git@gitlab.science.gc.ca:user123/CanESM5.git (push) Now this setup is all great and good if you only care about the versions contained in *your fork*. However, you will likely want to be able to do things like: - update your code from the central repo, or - compare, merge code from another user's fork To do this, you will need to add new remotes! **To add new remotes across all submodules and the super-repo**, developers can manually navigate to each directory and run the raw ``git remote add ``, or to make things easier, they can utilize ``git sremote``. Specifically, **if we want to add the central (internal) repository as a remote**, noting that it belongs to the ``CanESM`` group, all the you need to run is: .. code-block:: text >> # at the super-repo level >> git sremote CanESM .. note:: replace ``CanESM`` with another git user/group name if you want to add a differnent remote. After that, in the super-repo (and each submodule), you should now see .. code-block:: text >> git remote -v origin git@gitlab.science.gc.ca:user123/CanESM5.git (fetch) origin git@gitlab.science.gc.ca:user123/CanESM5.git (push) CanESM git@gitlab.science.gc.ca:CanESM/CanESM5.git (fetch) CanESM git@gitlab.science.gc.ca:CanESM/CanESM5.git (push) .. note:: if you would prefer to reference this new remote with a different name (as opposed to ``CanESM``), add the desired name as the second argument to ``git sremote``. Fetching changes from another remote """""""""""""""""""""""""""""""""""" The easily fetch changes across all repos, we utilize the ``git sfetch`` command. By default, it fetches changes from ``origin``, however you can easily fetch changes from a specific remote by providing arguments to ``git sfetch``. For example, to fetch changes from the remote we added :ref:`above`, we would execute: .. code-block:: git sfetch CanESM if we want to update the remote references for *all* branches that live on the main repo. **If we only care about one branch**, we can add a second argument, i.e. .. code-block:: git sfetch CanESM develop_canesm will only fetch updates to ``develop_canesm``. Comparing across remotes """""""""""""""""""""""" Assuming you've already added the necessary remote, you can easily compare your branches across remotes by simply executing .. code-blocK:: git diff / in each repository. For example, if we want to compare our current branch to ``develop_canesm`` on the central repo, we would execute .. code-block:: git sfetch CanESM develop_canesm # if not already done git diff CanESM/develop_canesm Merging in updates from other remotes """"""""""""""""""""""""""""""""""""" To illustrate how to merge in changes from another remote, lets consider the case where we want to update our local version of ``develop_canesm``, with that from the central repository, which has been added as a remote with the name ``CanESM`` (see :ref:`here`). All that needs to be done is .. code-block:: git scheckout develop_canesm # checkout your local version git sfetch CanESM develop_canesm # update what your repo knows about the central develop_canesm git smerge CanESM/develop_canesm develop_canesm # merge! The ``push`` the updated branch to your fork via .. code-block:: git spush origin develop_canesm # assuming your fork's remote name is set to 'origin' .. warning:: While merging is **required** by developers, it *can go sideways*. To reduce the chance for problems, it is recommended that: 1. developers use ``git status`` and ``git log`` to confirm that their repo is in the expected state **before** executing the merge and, 2. developers use ``git status`` **after** the merge to confirm that the merge executed correctly and completed before performing any other ``git`` operations For the general case, where a developer might want to merge in changes from ``branchA`` on remote ``remote2``, onto their local ``branchB``, the sequence of commands would look like: .. code-block:: git scheckout branchA git sfetch remote2 branchB git smerge remote2/branchB branchA Other Considerations -------------------- Allowing other developers/accounts to push to your fork """"""""""""""""""""""""""""""""""""""""""""""""""""""" Some users often end up using multiple accounts to interact with the same repository (i.e. those with "production" accounts). This has worked in the past, but now when users fork the desired repositories, they default to only having one member - the user who forked the repo - and thus, any account (other than that used to fork the repo) will not be able to push to the fork, and will get an error message like: .. code-block:: text GitLab: You are not allowed to push code to this project. fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. when they try to do so. **Fortunately, it is possible to give other accounts permission to push**, and this sections lays out how to do so, noting that this process only needs to be done 1 time / desired user / forked repo. With that said, to add additional accounts: 1. Navigate to the 'member's page from your repository dashboard (the repo's homepage) by selecting the gear in the upper right and clicking on "Members" 2. Add the desired account id by searching for it in the field under "Add new user to..." and select the desired account 3. Change the permission to "Developer" 4. Select "Add to Project" and thats it! **Again, this need to be done for each repo.** (Internal Use Only) Launching runs from your fork """"""""""""""""""""""""""""""""""""""""""""""""" As stated :ref:`here `, all production runs are still required to happen from the main repository, after user updates have been merged into ``develop_canesm``. However, during the development process, developers will want to test the changes on their forks. Fortunately, the process for doing so remains almost identical to that laid out :ref:`here `, except that an additional argument must be given to ``setup-canesm``. Specifically, the remote url must be given via the ``repo=`` argument, for example: .. code-block:: text setup-canesm runid=myrun config=ESM ver=mybranch repo=git@gitlab.science.gc.ca:sci123/CanESM5.git After this one difference, the rest of the instructions remaing the same. .. note:: If the ``repo=`` flag is not given to ``setup-canesm``, it defaults to using ``repo=git@gitlab.science.gc.ca:CanESM/CanESM5.git``.