Postprocessing: Difference between revisions
(→Ncview: added caviot that it cant plot icon grids) |
(repaired Icontools link) |
||
(8 intermediate revisions by 2 users not shown) | |||
Line 12: | Line 12: | ||
Search for the following lines: |
Search for the following lines: |
||
< |
<syntaxhighlight lang=bash># Path to the list with variables (is overwritten when -l specified) |
||
# Assumed to be on the same path as script |
# Assumed to be on the same path as script |
||
path_list=$SCRIPTPATH/list </ |
path_list=$SCRIPTPATH/list </syntaxhighlight> |
||
Change the name of the list to: |
Change the name of the list to: |
||
< |
<syntaxhighlight lang=bash>path_list=$SCRIPTPATH/samoa_list_icon-art </syntaxhighlight> |
||
Now you may use SAMOA with the ICON-ART output file out.nc with the following command: |
Now you may use SAMOA with the ICON-ART output file out.nc with the following command: |
||
< |
<syntaxhighlight lang=bash>./samoa.sh out.nc</syntaxhighlight> |
||
For all options see: |
For all options see: |
||
< |
<syntaxhighlight lang=bash>./samoa.sh --help</syntaxhighlight> |
||
⚫ | |||
== Data Processing == |
|||
⚫ | |||
Some ways to handle and modify the outputs of ICON-ART are described here. |
|||
=== CDO === |
|||
[https://code.mpimet.mpg.de/projects/cdo CDO] is a command line tool to interact with netCDF files, enabling to list the variables and do calculations efficiently. |
|||
Different examples for its usage can be found in the [https://code.mpimet.mpg.de/projects/cdo/wiki/FAQ CDO FAQ] |
|||
=== ICONTOOLS === |
|||
[https://c2sm.github.io/tools/icontools.html ICONTOOLS] is a command line tool used to produce the required external parameters data of a simulation and is especially good at remapping data from one grid to another. |
|||
⚫ | |||
⚫ | There are many ways to visualize data produced by ICON-ART. In general, there are two possibilities: The output may exist on the ICON grid or it may exist on an interpolated longitude/latitude grid. This can be chosen by adaptions of the output namelist. Although it comes along with a loss in information, sometimes it is only possible to use interpolated output. By this, the visualization is much easier to handle. |
||
In the following sections |
In the following sections some tools are introduced which can be used to visualize ICON output. Note, that '''only NETCDF''' is supported by ICON-ART so far. With the tool Ncview it is very easy to have a quick look into the interpolated model output. With ParaView and Met3D a nice-looking three-dimensional visualization can be created. Python in the recent years has become the standard tool for 2D visualisation and its capabilities far exceed those of Ncview. |
||
=== Ncview === |
=== Ncview === |
||
Line 37: | Line 46: | ||
Please note that Ncview does only work for latitude-longitude grid data and cannot be used for RXXBXX-style ICON grids without remapping. |
Please note that Ncview does only work for latitude-longitude grid data and cannot be used for RXXBXX-style ICON grids without remapping. |
||
=== |
=== Met3d === |
||
Met3D is a visualization software that can be used on a HPC system to visualize ICON as well as ICON-ART data in 3D. An online documentation can be found [https://met3d.wavestoweather.de/met-3d.html here]. Met3D needs remapped ICON fields to Latitude-Longitude . Furthermore, it is important to have the pressure (pres - variable) as model output, as Met3D transforms model levels in pressure levels for plotting. |
|||
"NCL is an interpreted language designed specifically for scientific data analysis and visualization. Portable, robust and free, NCL is available as binaries or open source." (https://www.ncl.ucar.edu/) |
|||
=== ParaView === |
=== ParaView === |
||
Line 46: | Line 57: | ||
ParaView was developed to analyze extremely large datasets using distributed memory computing resources. It can be run on supercomputers to analyze datasets of exascale size as well as on laptops for smaller data." (http://www.paraview.org/) |
ParaView was developed to analyze extremely large datasets using distributed memory computing resources. It can be run on supercomputers to analyze datasets of exascale size as well as on laptops for smaller data." (http://www.paraview.org/) |
||
Paraview can be used for both Latitude-Longitude grids as well as RXXBXX-style ICON grids. |
|||
=== Python === |
=== Python === |
||
Line 56: | Line 69: | ||
is friendly & easy to learn; |
is friendly & easy to learn; |
||
is Open.</pre> |
is Open.</pre> |
||
Using Python is a simple but effective way to display ICON-ART model output data. There is a large number of Packages available to help with Visualisation, the most useful Packages for visualising ICON-ART data are given in |
Using Python is a simple but effective way to display ICON-ART model output data. There is a large number of Packages available to help with Visualisation, the most useful Packages for visualising ICON-ART data are given in this [[#tab:pythonpackages|Table]] |
||
<div id="tab:pythonpackages"> |
<div id="tab:pythonpackages"> |
||
Line 70: | Line 83: | ||
|align="center"| xarray |
|align="center"| xarray |
||
|align="center"| reading in and processing netcdf datasets |
|align="center"| reading in and processing netcdf datasets |
||
|- |
|||
|align="center"| cartopy |
|||
|align="center"| Include country borders in plots |
|||
|- |
|- |
||
|align="center"| ... |
|align="center"| ... |
||
Line 75: | Line 91: | ||
|} |
|} |
||
Python can be used for both Latitude-Longitude grids as well as RXXBXX-style ICON grids. |
|||
</div> |
</div> |
Latest revision as of 17:12, 8 January 2024
Output Checks with SAMOA
SAMOA performs a sanity check on all model outputs that can be read by CDO. It checks if a variable lies in-between a predefined range and if the minimum and maximum value of each variable are the same. For this purpose CDO version 1.6.2rc3 is required currently (see https://code.zmaw.de/projects/cdo).
For more information about the usage please refer to the README-file within the SAMOA package. You can get a copy of the SAMOA script by writing an e-mail to the contact person of the ART code (see http://icon-art.imk-tro.kit.edu). SAMOA is licensed under the GNU GENERAL PUBLIC LICENSE Version 3.
As SAMOA is primarily developed for the usage with COSMO-ART and COSMO-CLM, you have to do a minor change before using it. The latest version of SAMOA has a list for the usage of SAMOA with ICON-ART output included but not loaded automatically. This list is called samoa_list_icon-art. You have to replace the default (COSMO) list that is used by SAMOA by editing samoa.sh:
Search for the following lines:
# Path to the list with variables (is overwritten when -l specified)
# Assumed to be on the same path as script
path_list=$SCRIPTPATH/list
Change the name of the list to:
path_list=$SCRIPTPATH/samoa_list_icon-art
Now you may use SAMOA with the ICON-ART output file out.nc with the following command:
./samoa.sh out.nc
For all options see:
./samoa.sh --help
Data Processing
Some ways to handle and modify the outputs of ICON-ART are described here.
CDO
CDO is a command line tool to interact with netCDF files, enabling to list the variables and do calculations efficiently. Different examples for its usage can be found in the CDO FAQ
ICONTOOLS
ICONTOOLS is a command line tool used to produce the required external parameters data of a simulation and is especially good at remapping data from one grid to another.
Visualisation
There are many ways to visualize data produced by ICON-ART. In general, there are two possibilities: The output may exist on the ICON grid or it may exist on an interpolated longitude/latitude grid. This can be chosen by adaptions of the output namelist. Although it comes along with a loss in information, sometimes it is only possible to use interpolated output. By this, the visualization is much easier to handle.
In the following sections some tools are introduced which can be used to visualize ICON output. Note, that only NETCDF is supported by ICON-ART so far. With the tool Ncview it is very easy to have a quick look into the interpolated model output. With ParaView and Met3D a nice-looking three-dimensional visualization can be created. Python in the recent years has become the standard tool for 2D visualisation and its capabilities far exceed those of Ncview.
Ncview
"Ncview is a visual browser for netCDF format files. Typically you would use ncview to get a quick and easy, push-button look at your netCDF files. You can view simple movies of the data, view along various dimensions, take a look at the actual data values, change color maps, invert the data, etc." (http://meteora.ucsd.edu/~pierce/ncview_home_page.html)
Please note that Ncview does only work for latitude-longitude grid data and cannot be used for RXXBXX-style ICON grids without remapping.
Met3d
Met3D is a visualization software that can be used on a HPC system to visualize ICON as well as ICON-ART data in 3D. An online documentation can be found here. Met3D needs remapped ICON fields to Latitude-Longitude . Furthermore, it is important to have the pressure (pres - variable) as model output, as Met3D transforms model levels in pressure levels for plotting.
ParaView
"ParaView is an open-source, multi-platform data analysis and visualization application. ParaView users can quickly build visualizations to analyze their data using qualitative and quantitative techniques. The data exploration can be done interactively in 3D or programmatically using ParaView’s batch processing capabilities.
ParaView was developed to analyze extremely large datasets using distributed memory computing resources. It can be run on supercomputers to analyze datasets of exascale size as well as on laptops for smaller data." (http://www.paraview.org/)
Paraview can be used for both Latitude-Longitude grids as well as RXXBXX-style ICON grids.
Python
On the official Website Python describes itself as
Python is powerful... and fast; plays well with others; runs everywhere; is friendly & easy to learn; is Open.
Using Python is a simple but effective way to display ICON-ART model output data. There is a large number of Packages available to help with Visualisation, the most useful Packages for visualising ICON-ART data are given in this Table
numpy | predefined Mathematical functions |
matplotlib | Plotting framework |
xarray | reading in and processing netcdf datasets |
cartopy | Include country borders in plots |
... |
Python can be used for both Latitude-Longitude grids as well as RXXBXX-style ICON grids.