Comments (8)
@MichaelSchulzMETNO @jgriesfeller @AugustinMortier @hannasv feel free to comment or make suggestions. I would like to have a first version ready for v0.8.1 and will try to work on it a little in Abisko, if there is spare time.
from pyaerocom.
@jgliss In the aerocom-tools the time frame for climatology is static (years 2000 to 2010) across all data sets. I think there should just be one climatology, maybe with different time frames per data set.
The constraints are currently at least 30 days of data within these 10 years.
Regarding the year: a climatology consists of just one year of data (2999 in case of the aerocom-tools).
In IDL it is possible to compare different years (e.g. year 2008 model and year 2005 obs), but the time co-location is not time aware (uses just indexes)..
from pyaerocom.
@jgriesfeller thanks for the helpful input. I agree that a fixed climatology should be used by default but the code should be implemented such that the default range can be changed by the user. And as you suggest, I think it is good to set the time frame per dataset.
Regarding the constraints, maybe we can be a little stricter and require at least a certain number of points per year?
Regarding the year: The colocation routines in pyaerocom are flexible w.r.t. time and can also colocate multiple year model data with observations.
E.g. if the input model data is AOD from 2005 to 2015 then the the obsdata (e.g. AERONET) is extracted within this time interval and colocated. Now, if the climatological colocation is active, then the obsdata would be a single year. In this case I see currently 2 options:
- Restrict the climatological colocation routine to single year model datasets.
- Or colocate each year in a multi-year model dataset with the same climatology.
from pyaerocom.
I suggest we aim now for a new standard climatology 2005-2014 as default.
For comparison it would be nice to also have the old mean 2000-2009 as Jan described it.
Monthly data should be constructed for each station. For now we should stick to the old definition, that you need 30 days in 10 years within a given month. (weekly samples, eg deposition, rain concentration, count as 7 days). Time colocation should happen on month basis then (not years, days etc). Models can be compared both from individual years and based on model climatolgies. However, for now we aim to compare sinle years from the model, eg 2010 CTRL-AP3 to the observed climatologies.
from pyaerocom.
@MichaelSchulzMETNO @jgriesfeller Are we applying any further constraints regarding the sampled years?
For instance, if there is a station, that has only data within 2009-2011, but in each of the months enough data, so that the "30 days in 10 years within a given month" criterium is fulfilled: then we would get a climatological timeseries also for this station, even though the available years do not necessarily represent 2005-2015.
We could for instance,
- require that temporal coverage must cover at least 2006-2014?
- And / or a minimum number of years (e.g. 5) in which a given month needs to be sampled (i.e. in order to get month X in the climatological time series, we need data for that month in at least 5 of the 10 considered years.
I would opt for both, probably...
from pyaerocom.
Hi, I tested the new feature in a new experiment that evaluates all CTRL models I use for the optics study against climatologies (2005-2015) from the ground-based observations used in (PIII-optics2019). The results are available here:
https://aerocom-evaluation.met.no/overall.php?project=aerocom&exp=PIII-optics2019-CLIM
At a first glance, there are differences, but they don't seem to be too large. E.g. bias in surface scattering changes by up to 12% as compared with the results that were colocated using 2010 observations (e.g. CAM5-ATRAS). For absorption coeffs. and AOD the differences are less pronounced (up to ~5%).
from pyaerocom.
This issue is stale because it has been open for 365 days with no activity. This issue will be closed in 14 days if no action is taken.
from pyaerocom.
This issue was closed because it has been inactive for 14 days since being marked as stale.
from pyaerocom.
Related Issues (20)
- Testdata minimal should be user-specific
- EBAS reader: open sqlite3 db in read-only mode only
- illogical signatures and input tests in mscw/addtional-variables HOT 1
- Update caching strategy for ungridded data HOT 3
- Mandatory data_revision on all obs-readers, disable last-file check in caching
- DataCoverageError when running pyaeroval ana.run with var_list and multiple obs-setups
- MSCW-reader reads in trends of all available year rather than only years selected with periods HOT 1
- add_maps tries to read min-maximum from model even if min-max is configured in glob_defaults.py
- Rename coarse to pm10 and fine to pm25 HOT 1
- pyaeroval: splitting of pyaerocom/aeroval and pyaerocom HOT 2
- Define ranges by frequency in var_scale_colmap
- Incorrect map frequencies: ModelMapsSetup's maps_freq is not being picked up in ModelMapsEngine::_process_map_var
- Model maps shown for EMEP should use the frequency of the Base_*.nc file rather than the colocated data and then resampling HOT 2
- MDA8 calculation for single year results in ts of two years
- MDA8 daily values have wrong timestamp
- MDA8 values not shown on Overall Evaluation HOT 3
- pyaerocom/aeroval output should use aerovaldb-locking for all output
- Milestones are removed when issues are added from the AeroTools/Backlog HOT 2
- Optimize performance of _process_statistics_timeseries HOT 10
- Pure hourly runs with o3 crash because o3mda8 is generated but not useable (for hourly) HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyaerocom.