swimdrinkfish / opendata Goto Github PK

View Code? Open in Web Editor NEW

11.0 10.0 6.0 79 KB

Open data standard for automated exchange of recreational water quality data

Home Page: https://www.recreationalwater.ca/

License: MIT License

water-quality data-exchange water-quality-information health swimming beach

opendata's Introduction

Open Data Standard for Recreational Water Quality

Welcome! Swim Drink Fish is developing a standard for exchanging recreational water quality data.

We need your help.

Goals:

Create a single standard for presenting recreational water quality data, so that different people monitoring different waters can share their results
Make the standard open, so that everyone can see it and shape it
Ensure data is shared in a machine-readable format (i.e., not pdf, not csv, not RSS) so that sharing is easy and automated

We need your help to refine the standard for the automated exchange of recreational water quality data.

Read on, or visit our project website for more information.

Background

The purpose of recreational water quality monitoring is to protect public health from contaminated water. Even where water is tested, however, it can be very difficult for the public to get an answer to the question “is it safe to swim?”

Hundreds of counties, municipalities, park organizations and nonprofits regularly test water for indicator bacteria, algae, and/or other harmful pollutants. The data should be shared quickly (within 24 hours) and with as wide an audience as possible in order to help people avoid waterborne illnesses; yet, this rarely happens.

The public faces four common barriers when they try to understand the health of their beach or swimming hole:

Access. Much of the water quality information collected is never published. It lives in data silos that are not accessible to the public.
Ease of Access. Public water quality does exist, but it can often be hard to find. It may be buried on websites or phone hotlines that aren’t easy for people to locate.
Clarity. The data is presented in a way that is difficult for the general public to understand and usually differs from place to place.
Age. When data is shared, it is often very out of date. Many places, for example, still only publish annual or monthly testing results.

Lack of water quality information means that people - often unknowingly - swim in contaminated waters. Studies from the Environmental Protection Agency estimate the between 3%-8% of people experience acute gastrointestinal illness (AGI) after swimming. An estimated 3.5 million Americans get sick each year. In Canada, the number is approximately 400,000. Children are the most likely to get sick.

The main reason water quality is not shared more widely is because there is no standardized way to present data.

Since 2011, Swim Guide has spent thousand of hours each manually compiling water quality information from websites (some using legacy technologies), phone hotlines, and spreadsheets so that 1.5 million people can access beach water information. Meanwhile, regulators in different countries struggle to consolidate water quality results from different communities to inform policy and funding decisions.

That’s why we decided to develop an open data exchange standard for recreational water quality information.

An open data standard will increase data exchange, improve public awareness of water quality, and aid researchers trying to protect recreational waters. The initiative will reduce the number of people getting sick from contact with polluted water and increase the number of environmental protection plans for communities affected by pollution.

This project is funded by the Canadian Internet Registration Authority (CIRA.ca).

Authors

Chris Wilson ([email protected]) Surfrider Foundation

Rajbir Parmar ([email protected]) Environmental Protection Agency, Office of Research and Development

Adam Griggs ([email protected] ) River Network

Joan Yee ([email protected]) Alberta Health Services

Dylan Neild ([email protected])

Gabrielle Parent-Doliner ([email protected])

Krystyn Tully ([email protected])

Project timeline

Oct 2017: Expert workshop to lay the groundwork for the open data standard

Dec 2017: Draft consultation period begins

Feb 2018: Comment period ends

March 2018: Pilot projects begins

June 2018: Launch of Open Data Standard

Invitation to Comment on the Open Data Standard for Recreational Water Quality - Version 1

Current Status

The current version of this standard proposal is version 1.0.1. Any comments, discussion, and decisions relating to the v1.0.1 standard will be implemented in subsequent releases.

Current version (v1.0.1): https://github.com/swimdrinkfish/opendata/tree/master/v1.0.1

Instructions for commenting

We encourage you to contribute to this standard on Github: https://github.com/swimdrinkfish/opendata/

Comments can also be sent to: Gabrielle Parent-Doliner Swim Guide Program Manager ([email protected])

Github instructions

Search existing issues

Before creating a new discussion, please look through existing issues to see whether your questions has already been asked or your comments have already been raised. If an issue addressing your question, comment, or problem has already been created, please contribute.

Join an existing discussion or comment on existing issue

To join an existing discussion or comment on an existing issue add new comments at the bottom of a discussion/issue. You can contribute in text, or richer content and links using Markdown syntax.

Create a new discussion

You must have a Github account and be logged in to create a new discussion. Click on the "New Issue" button to create a new conversation.

Label a discussion

Once you have created a new issue, you can add a label. The label should reflect the discussion item. This will make it obvious to other users that it is meant for review and comment.

Add a mention in a discussion

If you want to direct your comment, question or issue to a specific person you can add a mention in the text of your discussion or comment (type the '@' symbol and adding the person’s GitHub user name).

Use Markdown syntax

This forum supports the use of Github flavored Markdown syntax.

More info on the how-tos of contributing? Click here

opendata's People

Contributors

Stargazers

Watchers

Forkers

fooway stevenblack patrickleclair-gordonfn rowbottomn dylanneild shanstagram88

opendata's Issues

Add new method code - M-TEC-Ig agar method

This is a membrane filtration method (CFU)
Name is M-TEC-Ig agar method
alternate name is MECP E3371

description : Public Health Ontario method: Specimens are tested by the Membrane Filtration method modified from MECP E3371: A Membrane Filtration Method for the Detection and Enumeration of Total Coliform, Escherichia coli, and Enterococci/Fecal Streptococci in Environmental Samples for the microbiological indicator Escherichia coli (E. coli).

Remove minItems property from #/properties/records/

MinItems property for records Array does not allow for a schema compliant way to show that no records have been returned. This could happen when working with APIs, and there are no records responsive to a query.

This violates spec:
{ "$schema": "https://raw.githubusercontent.com/swimdrinkfish/opendata/master/v1.0.1/schema.json#", "documentTime": "2020-09-02T18:33:28.095921+00:00", "records": [] }

add cyanotoxins to substance codes

Appendix 2 - Substance Codes should include two other cyanotoxins: Anatoxin-a and Saxitoxin.

Update Advisory object with conditions, and add pass/fail object

Advisories can only be issued by authoritative bodies, such as an environmental health officer. Therefore suggest the following in order to accommodate sample results from bodies without authority to issue health advisory such as citizen science groups, research bodies, etc.

Add the following objects to the advisory object reference:

advisoryIssuedBy: (string); name of individual or body with authority to issue the advisory
result_passed: mandatory, enum (An object indicating yes or no if the described water quality sample is in excess of the prevailing local standard for recreational water use)

Investigate the possibility of “no status” updates

Investigate the possibility of adding a “no status available” update to the standard, thus allowing “no data” indicators. This would let monitoring authorities and labs indicate that there have been issues with sample data collection and/or decisions to skip collection for various reasons.

We presently depend on “no status is no status”, which effectively means that if data isn’t published it’s not available. The problem is that this requires (for example) a day to end before you know for sure that data for that day wasn’t available, and even then late delivery of data for unspecified delays is possible.

An explicit “there will be no data” indicator may be helpful.

Add new method - ASTM D6503 - 99 (Enterolert)

Add new multitube fermentation method ASTM D6503 - 99 (Idexx Enterolert). This is not an EPA method, rather an ASTM method.

Addition of method code "unknown"

Method code is a string enumeration value indicating the method code that was used to enumerate the sample result number. This method code should correspond to a method used to enumerate the sampled substance. For more information about the valid method codes, please see “Appendix 1 - Method Codes” in reference doc.

Suggest addition of method code "unknown/not provided" to the list of method codes.
"Unknown/not provided" will allow for data where the sample analysis method was not provided.

add water temperature objects

Recommend adding water temperature as a non-mandatory metadata in Sample Object Reference for version 1.1 of the standard.

As a best practice, monitors collect the temperature at each sample site. Important and useful piece of information.

Suggest the following temperature objects:

Temperature (field type: numeric)
temperatureUnit (field type : enum ; C for celsius; F for fahrenheit; K for Kelvin)

Additional Method Codes welcome

When we created the method code list for methods used to enumerate sample results we anticipated that, as new methods become available, the list would be updated. We invite everyone to review the list of methods and to contribute any we may have missed.
For example, we recently added two qPCR methods for E.coli: the EPA's draft Method C, and BioGX

Add new method for Cyanobacteria and toxins detection

Method 701.0 for microcystin should be added to Cyanobacteria and toxins detection methods

This method is used for the determination of total (extracellular and intracellular) microcystins –
ADDA in surface water, ground water and finished drinking water.
Reporting Limit (RL): 0.30 µg/L

https://epa.ohio.gov/Portals/28/documents/labcert/Ohio%20EPA%20DES%20701.0%20Version%202.2.pdf

Mechanism for revocation of a published sample result

I think we need to add a mechanism to the standard to revoke a published sample result if it is discovered after-the-fact that a result is not valid due to an issue like malfunctioning/miscalibrated test equipment, improper collection technique that may have resulted in contamination, or some other issue.

"substance": "clarity",
"method": "1337_turbidity_disk",
"units": "m",
"result": 1.5