Aeroval maps of PM seem wrong about pyaerocom HOT 13 CLOSED

Works with iris 3.4.1

from pyaerocom.

Just to add to this... The following maps should show the same (I created some monthly mean data with cdo):

The units are unfortunately not the same, but even the patterns look different. The files look OK. Very weird.

from pyaerocom.

Maybe it's just a fluke with lustre and if we ran the experiment again it might work?

from pyaerocom.

Processing model maps for IFS-ESUITE (concpm10)
Searching database for ECMWF_ESUITE
/lustre/storeB/project/aerocom/aerocom-users-database/ECMWF/ECMWF_ESUITE has subdir renamed. Using that one
Found match for ID ECMWF_ESUITE
Did not find concpm10 field but sconcpm10. Using the latter instead
Invalid time dimension coordinate in file:
/lustre/storeB/project/aerocom/aerocom-users-database/ECMWF/ECMWF_ESUITE/renamed/aerocom3_ECMWF_ESUITE_sconcpm10_Surface_2022_daily.nc.
Error: repr(Cube does not contain time dimension)

from pyaerocom.

But this is from all comps, ie.:

from pyaerocom.

Yes, just making notes to myself. This is typical although look suspicious at first glance

from pyaerocom.

Iris cube after rearranging longitude coords to [180, 180) seems fine

from pyaerocom.

the iris cube before being written to geojson. Between pyaerocom/io/iris_io.py(208)check_and_regrid_lons_cube()
and pyaerocom/aeroval/modelmaps_engine.py(158)_process_map_var() the problem occurs

Trackback...

(Pdb) traceback.print_stack()
  File "/lustre/storeB/users/lewisb/Python/Evaluations/aeroval/config/config_files/cfg_fix_864.py", line 428, in <module>
    res = ana.run()
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/aeroval/experiment_processor.py", line 130, in run
    engine.run(model_list=model_list, var_list=var_list)
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/aeroval/modelmaps_engine.py", line 45, in run
    files = self._run_model(model, var_list)
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/aeroval/modelmaps_engine.py", line 85, in _run_model
    _files = self._process_map_var(model_name, var, self.reanalyse_existing)
  File "/lustre/storeB/users/lewisb/Python/Evaluations/pyaerocom/pyaerocom/aeroval/modelmaps_engine.py", line 158, in _process_map_var
    contourjson = calc_contour_json(data, cmap=varinfo.cmap, cmap_bins=varinfo.cmap_bins)

from pyaerocom.

iris cube before being put into GriddedData object. Still looks okay

from pyaerocom.

After becoming a GriddedData object it gets messed up.

pyaerocom/io/readgridded.py line 2123.

data = GriddedData(
    input=cube,
    from_files=from_files,
    data_id=self.data_id,
    concatenated=is_concat,
    convert_unit_on_init=try_convert_units,
    **meta,
)

There are unit conversions occurring so look into that

from pyaerocom.

Before calling iris.grid.convert_units():

After calling iris.grid.convert_units():

Happens in pyaerocom/gridded_data.py::convert_unit() line 849. Seems like the issue is with iris

from pyaerocom.

BTW, in pyaerocom/io/iris_io.py line 535 has cube.attributes["timedim-corrected"] = True which is technically wrong because netcdf can't have boolean attributes when saving an iris cube.

from pyaerocom.

Using iris 3.5.0 pyhd8ed1ab_0 conda-forge. Iris 3.5.0 was released April 27th. 3.4.1 was released Feb 21

from pyaerocom.