e3sm-project / chemdyg Goto Github PK

@hsiangheleellnl and @tangq: I'm working on modifying zppy plugins to load and use external python modules. But I think the ChemDyg code needs more restructuring before I can go any further.

I was able to install chemdyg in my conda development environment:

git clone [email protected]:E3SM-Project/ChemDyg.git
cd ChemDyg
pip install .

I can successfully load the module

python
>>> import chemdyg

However, I cannot import the top level function that zppy will need to call. That function should be called'chemdyg, but it currently called e3sm_chem_diags. Also, it needs to be importable using something like

from chemdyg import chemdyg

Given the importance of NOx in reactive atmospheric chemistry, it would be useful to add a diagnostic that evaluates simulated NOx concentrations against observations.

The most suitable dataset for this purpose is likely the NOx observational dataset from the AToM campaign.

Ozone doesn't have surface emissions, so such a term in the summary table should be removed.

This was discussed at the infrastructure group meeting and documented at this meeting notes page.

The suggestions are:

• Use the centralized data repo at /lcrc/group/acme/public_html/diagnostics/observations/
• Datasets to be distributed with mache to all machines.
• Need to add time stamp in the input file names to differentiate versions

@hsiangheleellnl , as we talked the other day, it would be good to add the option of storing the data used in the plots or tables in netCDF (or other format) files.

For some sets (e.g., chemdyg_noaa_co_comparison), a job will be submitted even when the corresponding status file is "OK". This should be fixed.

It seems that ChemDyg followed the example of:
https://github.com/E3SM-Project/zppy/blob/c3f70dc8c814c36dcad5c380b1f86c8475a6192e/zppy/templates/e3sm_diags.bash
in creating python scripts within bash scripts.

This is not a very good software practice, since it's pretty confusing to debug and contribute to. Instead, zppy has a better example to work from:
https://github.com/E3SM-Project/zppy/blob/c3f70dc8c814c36dcad5c380b1f86c8475a6192e/zppy/templates/global_time_series.bash
https://github.com/E3SM-Project/zppy/blob/c3f70dc8c814c36dcad5c380b1f86c8475a6192e/zppy/templates/coupled_global.py

The bash script calls the python script:
https://github.com/E3SM-Project/zppy/blob/c3f70dc8c814c36dcad5c380b1f86c8475a6192e/zppy/templates/global_time_series.bash#L90
with several arguments. These arguments are then taken from the parameters array and copied into python variables:
https://github.com/E3SM-Project/zppy/blob/c3f70dc8c814c36dcad5c380b1f86c8475a6192e/zppy/templates/coupled_global.py#L491-L502

An even better practice would be to use the argparse package:
https://docs.python.org/3/library/argparse.html
Here is an example from e3sm_diags of how that can be used:
https://github.com/E3SM-Project/e3sm_diags/blob/833d9b2df0515565bf4150b20101e80dea647294/auxiliary_tools/metrics_checker.py#L21-L42

Once #1 is taken care of, I would be happy to provide an example for one of the existing .bash templates that you all could use to make similar changes to the others.

Have to manually change the files to be world-readable under the www directory. This is a pain - The permissions should be automatically set by the chemdyg scripts.

Plots like O3 hole are not created for F1850/F2010 cases.

The www directory is assumed to be ${www}/${case}/e3sm_chem_diags/plots/ and hence will be overwritten if running chemdyg for different time periods of the same simulation. This causes problem when long (e.g., 500 years) simulations need to run diagnostics for a certain time interval (e.g., 50 years).

It looks like the version number is still 0.1.4:
https://github.com/E3SM-Project/ChemDyg/blob/v0.1.5/setup.cfg#L3

Please make sure to update the version number before tagging a release.

The current implementation assumes that the O3 hole related plots always create from the beginning of a simulation. The users should be able to specify the metrics starting from the middle (e.g., skipping the spin-up).