Comments (9)
avaldebe
commented on July 26, 2024
1
should I update the metadata file?
I updated the metadata files on storeA/B, hope it helps
$ wc -l /lustre/store?/project/aerocom/aerocom1/AEROCOM_OBSDATA/EEA_AQeRep.NRT/download/metadata.csv
69274 /lustre/storeA/project/aerocom/aerocom1/AEROCOM_OBSDATA/EEA_AQeRep.NRT/download/metadata.csv
69274 /lustre/storeB/project/aerocom/aerocom1/AEROCOM_OBSDATA/EEA_AQeRep.NRT/download/metadata.csvfrom pyaerocom.
avaldebe
commented on July 26, 2024
Some of the 2024 stations are getting new data.
$ for F in GB_*; do tail -1 $F | cut -d, -f9,11,13-14; done | sort -u | 'grep' 2024-02
CO,hour,mg/m3,2024-02-05 03:00:00 +01:00
CO,hour,mg/m3,2024-02-06 16:00:00 +01:00
CO,hour,mg/m3,2024-02-06 19:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 01:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 02:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 12:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 13:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 14:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 16:00:00 +01:00
NO2,hour,µg/m3,2024-02-05 18:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 01:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 02:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 09:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 11:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 12:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 14:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 15:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 16:00:00 +01:00
NO2,hour,µg/m3,2024-02-06 19:00:00 +01:00
O3,hour,µg/m3,2024-02-05 14:00:00 +01:00
O3,hour,µg/m3,2024-02-05 15:00:00 +01:00
O3,hour,µg/m3,2024-02-05 18:00:00 +01:00
O3,hour,µg/m3,2024-02-06 01:00:00 +01:00
O3,hour,µg/m3,2024-02-06 11:00:00 +01:00
O3,hour,µg/m3,2024-02-06 12:00:00 +01:00
O3,hour,µg/m3,2024-02-06 14:00:00 +01:00
O3,hour,µg/m3,2024-02-06 15:00:00 +01:00
O3,hour,µg/m3,2024-02-06 16:00:00 +01:00
O3,hour,µg/m3,2024-02-06 17:00:00 +01:00
O3,hour,µg/m3,2024-02-06 19:00:00 +01:00
PM10,hour,µg/m3,2024-02-05 17:00:00 +01:00
PM10,hour,µg/m3,2024-02-06 14:00:00 +01:00
PM10,hour,µg/m3,2024-02-06 16:00:00 +01:00
PM10,hour,µg/m3,2024-02-06 19:00:00 +01:00
PM2.5,hour,µg/m3,2024-02-05 12:00:00 +01:00
PM2.5,hour,µg/m3,2024-02-05 13:00:00 +01:00
PM2.5,hour,µg/m3,2024-02-06 16:00:00 +01:00
PM2.5,hour,µg/m3,2024-02-06 19:00:00 +01:00
SO2,hour,µg/m3,2024-02-06 11:00:00 +01:00
SO2,hour,µg/m3,2024-02-06 14:00:00 +01:00
SO2,hour,µg/m3,2024-02-06 15:00:00 +01:00
SO2,hour,µg/m3,2024-02-06 16:00:00 +01:00
SO2,hour,µg/m3,2024-02-06 19:00:00 +01:00Should I re-download the 2022 and 2023 GB stations?
from pyaerocom.
avaldebe
commented on July 26, 2024
Should I re-download the 2022 and 2023 GB stations?
I downloaded all 2022 and 2023 GB stations and there is no new data.
from pyaerocom.
avaldebe
commented on July 26, 2024
looks like the station metadata has changed a lot since the last time it was updated
$ cd /lustre/storeB/project/aerocom/aerocom1/AEROCOM_OBSDATA/EEA_AQeRep.NRT/
$ wget http://discomap.eea.europa.eu/map/fme/metadata/PanEuropean_metadata.csv
$ wc -l renamed/metadata.csv PanEuropean_metadata.csv
67257 renamed/metadata.csv
69272 PanEuropean_metadata.csvfrom pyaerocom.
avaldebe
commented on July 26, 2024
should I update the metadata file?
from pyaerocom.
jgriesfeller
commented on July 26, 2024
The location of a stations is only noted in the metadata file. So it's vital to have that up to date. Without it, the station's data cannot be read.
pyaerocom/pyaerocom/io/read_eea_aqerep_base.py
Lines 678 to 682 in f8596b2
from pyaerocom.
jgriesfeller
commented on July 26, 2024
After @andagit complained again today about the missing stations, I checked the existing 17 stations for 2023 against the metadata file.
It turned out that all stations had a station classification of either urban-background or urban-traffic.
The problems is that in our config file we have the following station classification we accept:
"area_classification": [
"rural",
"rural-nearcity",
"rural-regional",
"rural-remote"
]
So basically none of the british stations match our classification criteria.
Another way of looking at it is that the UK does not submit rural stations anymore.
from pyaerocom.
andagit
commented on July 26, 2024
Just for the record - from EEA we got no. of stations from GB:
2015: 85, 2016: 77, 2017: 76, 2018: 70, 2019: 89, 2020: 34 (brexit,Corona), 2021: 90 (last v2), 2022: 17, 2023: 17
from pyaerocom.
andagit
commented on July 26, 2024
So could report back to ECMWF (Richard Engelen)/CAMS2_82 about the lack of GB stations.
from pyaerocom.
Related Issues (20)
- Testdata minimal should be user-specific
- EBAS reader: open sqlite3 db in read-only mode only
- illogical signatures and input tests in mscw/addtional-variables HOT 1
- Update caching strategy for ungridded data HOT 3
- Mandatory data_revision on all obs-readers, disable last-file check in caching
- DataCoverageError when running pyaeroval ana.run with var_list and multiple obs-setups
- MSCW-reader reads in trends of all available year rather than only years selected with periods
- add_maps tries to read min-maximum from model even if min-max is configured in glob_defaults.py
- Rename coarse to pm10 and fine to pm25 HOT 1
- pyaeroval: splitting of pyaerocom/aeroval and pyaerocom HOT 2
- Define ranges by frequency in var_scale_colmap
- Incorrect map frequencies: ModelMapsSetup's maps_freq is not being picked up in ModelMapsEngine::_process_map_var
- Model maps shown for EMEP should use the frequency of the Base_*.nc file rather than the colocated data and then resampling HOT 1
- MDA8 calculation for single year results in ts of two years
- MDA8 daily values have wrong timestamp
- MDA8 values not shown on Overall Evaluation HOT 3
- pyaerocom/aeroval output should use aerovaldb-locking for all output
- Milestones are removed when issues are added from the AeroTools/Backlog HOT 1
- Optimize performance of _process_statistics_timeseries HOT 7
- Pure hourly runs with o3 crash because o3mda8 is generated but not useable (for hourly)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyaerocom.