Comments (3)
lewisblake
commented on August 30, 2024
- @jgriesfeller needs this for the pyaerocom-plotting project
- says there is likely a collocated reader file already, matter of getting this to be read earlier in the pipeline
- starts as a specification problem. will need to read in collocated data objects created outside of pyaerocom, so we need to define exactly the specifications of these netCDF files. can start with a ncdump -h -ct on a collocated file to share with partners
from pyaerocom.
jgriesfeller
commented on August 30, 2024
Since I just started reading the files for pyaerocom_plotting:
Reading the files is easy:
from pyaerocom.colocateddata import ColocatedData
file = "/home/jang/data/aeroval-local-web/coldata/c3s//LTS_MODIS/LTS.SU/od550aer_od550aer_MOD-LTS.SU_REF-AeronetSunV3L2_20030101_20181231_daily_ALL-wMOUNTAINS.nc"
bla = ColocatedData(file)>>> type(bla)
<class 'pyaerocom.colocateddata.ColocatedData'>
Getting the workflow right is the bigger problem.
Note that the file above was written by aeroval
from pyaerocom.
lewisblake
commented on August 30, 2024
(pya) [lewisb@ppi-r8login-b1 IFS-Reanalysis-GHG]$ ncdump -h vmrco2_vmrco2_MOD-IFS-Reanalysis-GHG_REF-ICOS_20180101_20211231_daily_ALL-wMOUNTAINS.nc
netcdf vmrco2_vmrco2_MOD-IFS-Reanalysis-GHG_REF-ICOS_20180101_20211231_daily_ALL-wMOUNTAINS {
dimensions:
data_source = 2 ;
time = 1461 ;
station_name = 35 ;
variables:
string data_source(data_source) ;
int64 time(time) ;
time:units = "days since 2018-01-01 12:00:00" ;
time:calendar = "proleptic_gregorian" ;
string station_name(station_name) ;
double latitude(station_name) ;
latitude:_FillValue = NaN ;
latitude:standard_name = "latitude" ;
latitude:units = "degrees" ;
double longitude(station_name) ;
longitude:_FillValue = NaN ;
longitude:standard_name = "longitude" ;
longitude:units = "degrees" ;
double altitude(station_name) ;
altitude:_FillValue = NaN ;
double vmrco2(data_source, time, station_name) ;
vmrco2:_FillValue = NaN ;
string vmrco2:data_source = "ICOS", "ECMWF_I1DG" ;
string vmrco2:var_name = "vmrco2", "vmrco2" ;
string vmrco2:var_name_input = "vmrco2", "vmrco2" ;
vmrco2:ts_type = "daily" ;
vmrco2:filter_name = "ALL-wMOUNTAINS" ;
string vmrco2:ts_type_src = "hourly", "daily" ;
string vmrco2:var_units = "ppm", "ppm" ;
vmrco2:data_level = 3LL ;
vmrco2:revision_ref = "n/a" ;
string vmrco2:from_files = "aerocom3_ECMWF_I1DG_mmrco2_Surface_2018_daily.nc", "aerocom3_ECMWF_I1DG_mmrco2_Surface_2019_daily.nc", "aerocom3_ECMWF_I1DG_mmrco2_Surface_2020_daily.nc", "aerocom3_ECMWF_I1DG_mmrco2_Surface_2021_daily.nc" ;
vmrco2:from_files_ref = "None" ;
vmrco2:colocate_time = 0LL ;
vmrco2:obs_is_clim = 0LL ;
vmrco2:pyaerocom = "0.14.0" ;
vmrco2:CONV\!min_num_obs = "{\'monthly\': {\'daily\': 21}, \'daily\': {\'hourly\': 18}}" ;
vmrco2:resample_how = "mean" ;
vmrco2:model_name = "IFS-Reanalysis-GHG" ;
vmrco2:obs_name = "ICOS" ;
vmrco2:vert_code = "Surface" ;
vmrco2:diurnal_only = 0LL ;
vmrco2:coordinates = "latitude longitude altitude" ;
}from pyaerocom.
Related Issues (20)
- tests with lustre-access fail HOT 5
- Pya browse for ungridded observations is broken. HOT 2
- adjust URL for testdata
- Make aeroval json reading work with Pyaro HOT 1
- is this typo a bug? HOT 1
- is this a bug? or it is masking a bug elsewhere? HOT 1
- geodesy tests are broken or outdated HOT 1
- test_config.py fail on a machine with access to /lustre/storeA and /lustre/storeB
- Allow ordering of statistics HOT 1
- pytest should not fail on warinings HOT 1
- Remove warnings in tests
- pyaerocom gives warnings about files not being in metastandards version HOT 1
- Only read files fullfilling certain convention
- venv 3.12 tests on Ubuntu 22.04 are too slow HOT 4
- Dropping of statistics doesn't work when "num_valid" is 0
- CAMS2_82 new CNEMC reader - Chilean observations HOT 1
- Move AerovalConfigError and AerovalTrendsError into aeroval
- Implement ColocationSetup, ObsCollection, and ModelCollection classes with Pydantic
- GHOST data update
- Local tests not passing for test_config.py and test_geodesy.py HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyaerocom.