Giter Club home page Giter Club logo

Overview

This repository aims to provide a platform for thinking about, and developing, a unified view of metadata elements required to describe climate indices (aka climate indicators).

Currently, the main component is the spreadsheet file, master_table.xls, that contains several tables as separate sheets. The .xls format should [hopefully] be readable/editable by several common desktop office programs.

The repository also contains a small python package that can convert the spreadsheet into a set of equivalent yaml files that are better suited for automatic processing. For more information about the python program, take a look at the python sub-directory.

Contributing

We encourage contributions via discussions in issues. If you have a suggestion for the improvement of the metadata or the distribution of this work, please have a look at the issues. To help you starting a new issue there are three issue templates:

  • Request new climate index --- this template is divided into two parts, one for a general description of the new index, and one part for more precisely specifying entries in the index_definition speadsheet (see below). There is no need to complete all parts, just fill out those parts that you feel are relevant for your proposal.
  • Correction to index definition --- this template specifically refers to the index_definition spreadsheet (see below). Here we would like you to specify which table entries are wrong and how they should be changed.
  • General issue --- any other issue, question or discussion.

Don't hesitate to open a new one to initiate a discussion on any question or topic related to this effort!

Download

Standard github and git approaches apply when using the green button "Code" (above right), which does not include yaml files of the tables. To the right, under the heading "Releases" packaged versions of the repository are available as compressed files. These include yaml files that are up-to-date with the corresponding tables of the spreadsheet file. The compressed files are available both for MS-DOS/Windows line endings (.zip) and Linux line endings (.gz).

Approach

To facilitate data exchange and dissemination the metadata should, as far as possible, follow the Climate and Forecasting (CF) Conventions. Considering the very rich and diverse flora of climate indices this is however not always possible. By collecting a wide range of different indices it is easier to discover any common patterns and features that are currently not well covered by the CF Conventions.

With respect to climate index metadata the CF Conventions are in a sense both permissive and restrictive. This has some bearing on which indices are at all included in the list, and which metadata elements are present, as well as the (somewhat subjective) classification as ready or not.

Further details here (click here)

CF is permissive in the sense that only the bare essential information for understanding what the data represents are mandatory (cf. here, especially paragraph 8), and that any additional information can be included with few limitations. Thus, almost any climate index dataset can be published in a CF compliant way, but only with the bare minimum of standardised metadata. For such free (i.e., non-managed) information there are no rules, which means that it is difficult or intractable to develop common standardised workflows that would depend on this particular metadata information. But the CF Conventions also includes a range of more detailed metadata components that are managed according to specific rules. With these managed components it is possible to provide a richer and much more detailed description of the dataset. For these managed components CF is however often restrictive in that the rules are not always well suited to handle climate index metadata.

At the 2021 CF Workshop a plenary presentation outlined the links between metadata requirements for describing the climate indices and the CF Conventions (version 1.9).

In the index definition table the column ready (second left) subjectively indicates how complete the metadata description is. In general terms, indices marked as ready ("1") either have full metadata description, or there are advanced plans for what needs to be done. In particular, the focus is on the following elements of the CF Conventions:

  • standard_name (recommended if available).

  • long_name (free text, recommended).

  • unit (required if standard name is used, else recommended).

  • cell_methods (recommended if available).

Some relevant open issues at the CF github repository (click here)

More details (as per late 2020) regarding the approach towards structuring the climate index metadata is available in an IS-ENES3 project report M10.3 - Climate indicators/indices and file metadata specifications and tools.

This repository is in active development, and the content will frequently change.

File Format and Contents

Spreadsheet

The metadata is contained in the spreadsheet file master_table.xls and its sheets are as follows:

  • README --- explains contents and formatting of the spreadsheet file itself.

  • index_definitions --- the main table holding the metadata for the individual indices. Most of the indices developed by the ETCCDI and ET-SCI are included, as are the ones produced by ECA&D. However, some of the more complex indices remain to be included.

  • variables --- specification of input variables (following CMIP5/6 and CORDEX rules). This sheet also gives non-exhaustive lists of common aliases for the variable names. Its role is to provide a link from the actual variable name in the input file to the standardised variable names used in the index_definitions table. It is not intended to be prescriptive or restrictive regarding what variable names can be used or are suitable.

  • index_functions --- contains details about the calculation methods used for the indices. This is referred to in the index_definitions sheet.

  • ECA&D --- list of indices produced by ECA&D. Many of these are already covered by existing entries in the index_table sheet.

Yaml Files

The spreadsheets can be converted to a set of yaml files using the python package found in this repository. These are automatically generated and distributed alongside the spreadsheet in the metadata distributions linked to above.

The purpose of the yaml files is to give the same information as in the spreadsheet in a format that allows for a richer, hierarchical structure that also lends itself better to automatic processing. The downside is, that this makes it slightly more difficult to get a quick overview of all the available information.

At the moment only the index_definitions and variables sheets of the spreadsheet file are transformed into the corresponding index_definitions.yml and variables.yml files. The README sheet, insofar it gives information beyond the present document, only applies to the spreadsheet. The ECA&D sheet mainly exists to track the open indices of that collection until they are fully integrated into the main table index_definitions. Consequently, those two sheets are not likely to be converted in the future. The index_functions sheet will be added to the yaml files in the future.

Acknowledgement

This work is supported by the European Horizon 2020 programme through the projects IS-ENES3 (2019-2023) and CLIMATEUROPE2 (2022-2027), as well as by the SMHI Rossby Centre.

License

Clix-meta(c) 2020-2023 by Lars Bärring and Klaus Zimmermann, Rossby Centre, Swedish Meteorological and Hydrological Institute (SMHI).

The spreadsheet and all the metadata therein is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License.

You should have received a copy of the license along with this work. If not, see http://creativecommons.org/licenses/by-sa/4.0/.

Clix Meta's Projects

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.