Comments (14)
Currently, there is no way to specify the starting month and year that I am aware off. I made a proposal on issue #565 to solve this.
from esmvalcore.
I think there as an alternative solution, which is to use the time_slice
preprocessor (loaded as extract_time
). I just though it would be easier to add it into the dataset
description.
from esmvalcore.
Indeed this is simply not implemented yet. I think the most flexible and clear approach is more or less what @ledm proposed above, i.e. to allow specification at model level. I would propose to use
{
dataset: HadGEM2-CC
project: CMIP5
exp: historical
ensemble: r1i1p1
start_year: 1989
start_month: March
start_day: 3
end_year: 2004
end_month: October
end_day: 15
}
Specifically, writing out the names of the months is more clear than using a number for the month, because you never really know if 01-09-2001 means the September 1, 2001 or January 9, 2001.
from esmvalcore.
Alternatively, the more concise syntax proposed by @jvegasbsc is also fine with me:
{
dataset: HadGEM2-CC
project: CMIP5
exp: historical
ensemble: r1i1p1
start: 1989-03-03 10:24:00
end: 2004-10-15 11:32:00
}
where we allow any valid yaml datetime specification.
from esmvalcore.
@ledm if you are happy with the proposed solution please close this.
from esmvalcore.
I think we would need to choose and then implement the proposed feature before closing this issue.
At the moment, the only way to do this is with the preprocessor function and it will use the same settings for all datasets:
preprocessors:
profile:
extract_time:
start_year: 1989
start_month: 12
end_year: 2004
end_month: 3
from esmvalcore.
Both solutions are fine, but I also have a preference for the more concise syntax proposed by @jvegasbsc.
Just to make sure, the previous settings will remain valid too, right? We don't want to have to rewrite all the recipes to accommodate this solution.
While we're at it, would it possible to leave the dataset time
settings blank and load all available years? That would also be very useful!
Cheers,
Lee
from esmvalcore.
If we're going to change the way you specify the start and end date/time in the diagnostic from 'the current start_year
, end_year
to start
and end
as proposed here, I would recommend allowing only that way, because if there's two ways to do it, it is confusing. It isn't that much work to change the few recipes we have so far.
@mattiarighi What is your preferred solution? Option 1 or option 2?
from esmvalcore.
I find Option 1 better, also for the sake of backward compatibility.
Going for Option 2 would not only mean to change the recipes, but also a lot of diagnostics and some shared scripts, since many of the NCL scripts use the attributes start_year
and end_year
from dataset_info
in the interface file.
from esmvalcore.
@ledm Just to get a sense of the priority of this issue: does just specifying the start and end month in the preprocessor solve your issue? Or are the required start and end month different for each dataset?
Can you define a preprocessor like this:
preprocessors:
profile:
extract_time:
start_month: 12
end_month: 3
and let me know if it solves your problem?
from esmvalcore.
This has been my solution for the entire time. It certainly solved the problem, but it's not great to have to define the time range twice.
In addition, it really would be incredibly useful to have a "run over all available model time" option too. Here are some possible use cases:
- Every observational dataset will have a different time range.
- For monitoring ongoing model runs, run over all data would be more useful than looking at a strict pre-defined time range.
- In case there are some problems with CMIP6 data appearing on BADC in stages.
from esmvalcore.
In addition, it really would be incredibly useful to have a "run over all available model time" option too. Here are some possible use cases:
- Every observational dataset will have a different time range.
- For monitoring ongoing model runs, run over all data would be more useful than looking at a strict pre-defined time range.
- In case there are some problems with CMIP6 data appearing on BADC in stages.
Please open a separate issue to discuss this feature.
from esmvalcore.
Did we ever make a decision on this issue?
@bouweandela wrote:
Alternatively, the more concise syntax proposed by @jvegasbsc is also fine with me:
{ dataset: HadGEM2-CC project: CMIP5 exp: historical ensemble: r1i1p1 start: 1989-03-03 10:24:00 end: 2004-10-15 11:32:00 }where we allow any valid yaml datetime specification.
I agree that this would be a nice change, as long of the old method (specified by year only) was also supported.
This would be a nice change to the way that we load datasets by time range, notably for seasonal calculations, instead of just loading a range of years.
from esmvalcore.
This has been implemented some time ago, it can be done using timerange
.
from esmvalcore.
Related Issues (20)
- Improve the ESMValCore release documentation
- Automatically select a lazy regridding scheme in `regrid` preprocessor function for 2D lat/2D lon grids when possible HOT 6
- Drop support for Python 3.9 HOT 3
- Update Iris pin to avoid using versions with memory issues HOT 4
- Increase the version number for 2.11.0 release
- Add draft release notes for v2.11.0rc1 HOT 1
- Remove `compilers` from `environment.yml`
- Recipe test results for ESMValCore v2.11.0rc1 HOT 15
- Diagnostic failure for `recipe_martin18grl.yml` on `v2.11.0rc1` HOT 1
- Diagnostic failure for `recipe_pcrglobwb.yml` on `v2.11.0rc1` HOT 1
- Diagnostic failure for `recipe_lauer22jclim_fig3-4_zonal.yml` on `v2.11.0rc1` HOT 1
- Add native6, OBS6 and RAWOBS rootpaths to metoffice config-user.yml template, and remove temporary dir
- Add control over chunks at load time
- Interpolation in `resample_hours`
- Adding ETCCDI custom CMOR tables for HadEX3 CMORizer and IPCC diagnotics HOT 1
- Make extract_point/extract_location/extract_coordinate_points preprocessor functions lazy HOT 1
- Lazy derive preprocessor function
- [Numpy2] We need to unpin pandas (free :panda_face: )
- [Numpy2] Preprocessor `distance_metric` returns value 0 instead of `--` (masked element) with `numpy==2.0.0` for metrics that use `np.sqrt` or `da.sqrt` HOT 1
- Update the version number and release notes for v2.11.0rc2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from esmvalcore.