Getting Started: Difference between revisions

From icon-art guide
Jump to navigation Jump to search
(→‎Creating a Runfile: added info for the info file)
(11 intermediate revisions by 2 users not shown)
Line 3: Line 3:
As for most atmospheric models, it is strongly recommended to run ICON-ART on a High Performance Computing system such as Levante from the [https://www.dkrz.de/en DKRZ] or [https://www.scc.kit.edu/en/services/horeka.php HoreKA] from KIT. This usually requires an account which has to be obtained through the respective HPC Systems procedures.
As for most atmospheric models, it is strongly recommended to run ICON-ART on a High Performance Computing system such as Levante from the [https://www.dkrz.de/en DKRZ] or [https://www.scc.kit.edu/en/services/horeka.php HoreKA] from KIT. This usually requires an account which has to be obtained through the respective HPC Systems procedures.


== The auto-icon tool for automating runs ==
== Getting the source code ==
''auto-icon'' is a flexible tool for automating ICON(-ART) runs, starting from the retrieval of input data to post-processing and visualization. After an initial set-up, a great simplification in running various examples can be achieved. ''auto-icon'' takes care of obtaining the model code and installing it on the desired HPC system, retrieving parts of the input data (more to come) and running the model. Custom post-processing and visualization tasks can be added to the pipeline to get them executed automatically. The automation engine behind ''auto-icon'', called [https://autosubmit.readthedocs.io/en/master/ Autosubmit], is a powerful and flexible tool, providing multi-platform support, fault tolerance and an experiment database. The latter stores metadata for all conducted experiments and allows simple finding and reproduction of existing workflows, thus promoting the [http://www.go-fair.org/fair-principles/ FAIR] principles.


Especially if you are new to the ICON-ART model, ''auto-icon'' can help you set up a first run as it builds the model for you and has a bunch of example cases built in, running out of the box. These [https://www.icon-art.kit.edu/ ICON-ART standard cases] are included as templates along with a few other cases. There is a separate [https://gitlab.dkrz.de/auto-icon/auto-icon/-/wikis/home Wiki] for ''auto-icon'' with a [https://gitlab.dkrz.de/auto-icon/auto-icon/-/wikis/Shortest-guide-to-success shortest guide to success] and a [https://gitlab.dkrz.de/auto-icon/auto-icon/-/wikis/Usage/Step-by-step-guide more detailed guide].


The remainder of this Wiki serves as valuable source of information on how the ART module itself works, while the ''auto-icon'' Wiki only deals with technical aspects of ''auto-icon''.
A user who wants to work with ICON-ART has to sign the ICON license agreement with the German Weatherservice (DWD) and Max-Planck-Institute for Meteorology (MPI-M) first. Further information can be found on the following website:
Further, ''auto-icon'' does not use runscripts to conduct the runs but instead loads a configuration and a namelist describing the whole model workflow. The correspondence of runscript entries with the configuration is demonstrated in [https://gitlab.dkrz.de/auto-icon/auto-icon/-/wikis/Usage/Transform-a-runscript-into-an-auto-icon-configuration this guide].


For starting your ICON model run with ''auto-icon'', you go to one of the linked guides and skip the remainder of this page.
https://code.mpimet.mpg.de/projects/iconpublic


== Getting the source code ==
Afterwards you also should get access to the DKRZ gitlab. If you have access you can clone ICON.

The source code for ART is available in the open-source ICON repository under [http://www.icon-model.org www.icon-model.org]


Example for cloning the ICON-KIT repository including all submodules (including ART):
To clone the ICON repository use :


<code>
<code>
git clone --recursive git@gitlab.dkrz.de:icon/icon-kit.git
git clone --recursive https://gitlab.dkrz.de/icon/icon-model.git
</code>

This will get the ICON reository. To get the all submodules (including ART):

<code>
git submodule update --init
</code>
</code>


Line 21: Line 32:


ICON-ART is already included in the most recent ICON version. For Instructions on how to install ICON, please refer to the first chapter of the [https://www.dwd.de/DE/leistungen/nwv_icon_tutorial/nwv_icon_tutorial.html:official ICON Model Tutorial].
ICON-ART is already included in the most recent ICON version. For Instructions on how to install ICON, please refer to the first chapter of the [https://www.dwd.de/DE/leistungen/nwv_icon_tutorial/nwv_icon_tutorial.html:official ICON Model Tutorial].
The only caveat is that during the configuration step the tag <code> --enable-art </code> has to be included.
The only caveat is that during the configuration step the tag <code> --enable-art </code> has to be included. In addition, to set up and use chemical mechanisms using the MECCA/KPP the related interfaces have to be included using the tag <code> --enable-art —enable-art-gpl </code> (Be aware that you accept the GPL conditions for MECCA and KPP when you use this option).


<b>Step-by-step guide:</b>
<b>General step-by-step guide:</b>


* Navigate to your ICON main folder.
* Navigate to your ICON main folder.
Line 32: Line 43:
<b>Example for HoreKa at KIT:</b>
<b>Example for HoreKa at KIT:</b>
# Access your ICON main folder.
# Access your ICON main folder.
# Run the following command: <code>config/kit/hk.intel-2022-openmpi-4.0 --enable-art</code>
# Run the following command: <code>config/kit/hk.intel-2022-openmpi-4.0 --enable-art --enable-ecrad</code>
# Execute <code>make -j4</code>
# Execute <code>make -j4</code>
# You should now have a functional binary with ART integration. For other HPC systems, substitute the config script with the one relevant to your HPC system.
# You should now have a functional binary with ART integration. For other HPC systems, substitute the config script with the one relevant to your HPC system.
Line 38: Line 49:
== Creating a Runfile ==
== Creating a Runfile ==


* Go to <code>icon-kit/run/checksuites.icon-kit</code>
* Go to <code>icon-kit/run/checksuite.icon-kit</code>


* Run bash-script <code>run_testsuite</code> via <code>./run_testsuite</code>
* Run bash-script <code>run_testsuite</code> via <code>./run_testsuite</code>


* The script creates the folder runscripts, which contains exemplary runfiles which are adapted to some common HPC-Systems you might be using.
* The script creates the folder runscripts, which contains exemplary runfiles which are adapted to your HPC-System (if available in the config files). For the description of the runscripts see the table below.


* in <code>icon-kit/run/checksuites.icon-kit/Test-<current_date>.info</code> you will find a few informations to the ICON-ART Testsuite you just created, including your output directory when you perform the model runs in the next step
* in <code>icon-kit/run/checksuites.icon-kit/Test-<current_date>.info</code> you will find a few informations to the ICON-ART Testsuite you just created, including your output directory when you perform the model runs in the next step

* To run your chosen runscript just execute from the console, e.g. by typing <code>runscripts/NWP_LIFETIME.run</code>

{| class="wikitable"
|-
! runscript !! description
|-
| NWP_OH_CHEMISTRY.run || Short example for simplified oh chemistry
|-
| NWP_GASPHASE.run || Example for MECCA chemistry based on https://gmd.copernicus.org/articles/11/4043/2018/
|-
| NWP_LIFETIME.run || Example for parameterized chemtracer chemistry including lifetime, simnoy, linoz and passive tracers, as well as regional tracers and PSCs
|-
| NWP_EXT_DATA.run || tbd
|-
| NWP_LIFETIME_lart.run || tbd
|-
| NATAERO_NORAD.run || tbd
|-
| VOLAERO_RAD.run || tbd
|}


== Running a Job ==
== Running a Job ==

Revision as of 14:13, 9 October 2024

Requirements to run ICON-ART

As for most atmospheric models, it is strongly recommended to run ICON-ART on a High Performance Computing system such as Levante from the DKRZ or HoreKA from KIT. This usually requires an account which has to be obtained through the respective HPC Systems procedures.

The auto-icon tool for automating runs

auto-icon is a flexible tool for automating ICON(-ART) runs, starting from the retrieval of input data to post-processing and visualization. After an initial set-up, a great simplification in running various examples can be achieved. auto-icon takes care of obtaining the model code and installing it on the desired HPC system, retrieving parts of the input data (more to come) and running the model. Custom post-processing and visualization tasks can be added to the pipeline to get them executed automatically. The automation engine behind auto-icon, called Autosubmit, is a powerful and flexible tool, providing multi-platform support, fault tolerance and an experiment database. The latter stores metadata for all conducted experiments and allows simple finding and reproduction of existing workflows, thus promoting the FAIR principles.

Especially if you are new to the ICON-ART model, auto-icon can help you set up a first run as it builds the model for you and has a bunch of example cases built in, running out of the box. These ICON-ART standard cases are included as templates along with a few other cases. There is a separate Wiki for auto-icon with a shortest guide to success and a more detailed guide.

The remainder of this Wiki serves as valuable source of information on how the ART module itself works, while the auto-icon Wiki only deals with technical aspects of auto-icon. Further, auto-icon does not use runscripts to conduct the runs but instead loads a configuration and a namelist describing the whole model workflow. The correspondence of runscript entries with the configuration is demonstrated in this guide.

For starting your ICON model run with auto-icon, you go to one of the linked guides and skip the remainder of this page.

Getting the source code

The source code for ART is available in the open-source ICON repository under www.icon-model.org

To clone the ICON repository use :

git clone --recursive https://gitlab.dkrz.de/icon/icon-model.git

This will get the ICON reository. To get the all submodules (including ART):

git submodule update --init

Installation

ICON-ART is already included in the most recent ICON version. For Instructions on how to install ICON, please refer to the first chapter of the ICON Model Tutorial. The only caveat is that during the configuration step the tag --enable-art has to be included. In addition, to set up and use chemical mechanisms using the MECCA/KPP the related interfaces have to be included using the tag --enable-art —enable-art-gpl (Be aware that you accept the GPL conditions for MECCA and KPP when you use this option).

General step-by-step guide:

  • Navigate to your ICON main folder.
  • Within this directory, you will find the 'config' folder.
  • Inside the 'config' directory, there are several subfolders corresponding to different institutions.
  • In each institutional folder, you will find configuration scripts tailored for various computers and compilers.

Example for HoreKa at KIT:

  1. Access your ICON main folder.
  2. Run the following command: config/kit/hk.intel-2022-openmpi-4.0 --enable-art --enable-ecrad
  3. Execute make -j4
  4. You should now have a functional binary with ART integration. For other HPC systems, substitute the config script with the one relevant to your HPC system.

Creating a Runfile

  • Go to icon-kit/run/checksuite.icon-kit
  • Run bash-script run_testsuite via ./run_testsuite
  • The script creates the folder runscripts, which contains exemplary runfiles which are adapted to your HPC-System (if available in the config files). For the description of the runscripts see the table below.
  • in icon-kit/run/checksuites.icon-kit/Test-<current_date>.info you will find a few informations to the ICON-ART Testsuite you just created, including your output directory when you perform the model runs in the next step
  • To run your chosen runscript just execute from the console, e.g. by typing runscripts/NWP_LIFETIME.run
runscript description
NWP_OH_CHEMISTRY.run Short example for simplified oh chemistry
NWP_GASPHASE.run Example for MECCA chemistry based on https://gmd.copernicus.org/articles/11/4043/2018/
NWP_LIFETIME.run Example for parameterized chemtracer chemistry including lifetime, simnoy, linoz and passive tracers, as well as regional tracers and PSCs
NWP_EXT_DATA.run tbd
NWP_LIFETIME_lart.run tbd
NATAERO_NORAD.run tbd
VOLAERO_RAD.run tbd

Running a Job

For a user who succeeded in running the ICON model, there are only a few steps to run the ART extension along with the ICON model. A description how to run the ICON model can be found in .

In order to run ICON-ART, one has to do the following steps:

  • Make sure you have everything required for an ICON run
  • Prepare the input data (see section Input )
  • Inside the runscript in the namelist run_nml, set the main switch for ICON-ART to true: lart = .true.
  • Add a namelist art_nml and choose the namelist parameters for the ART setup as described in Input.
  • Adapt the XML files for tracers, emi. The number of tracers related to a specific setup is equal to the number of possible prognostic output fields as described in Input.
  • Add an output namelist as described in for the species you are interested in Input.
  • Submit the job analogous to an ICON job.