Giter Club home page Giter Club logo

ca-stormwater-data-challenge's Introduction

CA-Stormwater-Data-Challenge

The Urban Drool Tool:

An Inter-Agency Collaboration Between Moulton Niguel Water District, Orange County Public Works, and the California Data Collaborative

Introduction:

Our aim for this collaboration would be to gain insight on surface water drainage areas where altered dry weather water balance or flow regimes have been identified. Many of these stream reaches are impacted by unnatural, unpermitted, non-exempted dry weather flows (Urban Drool).

The ability to quantify and visualize the amount of Urban Drool contributing to discharges at stormdrain outfalls can help to determine the most appropriate and effective actions to restore natural water balance. The ongoing drought conditions in California have fostered the collaboration of multiple agencies and focused on the nexus of water resources as both a supply and environmental issue. Addressing systemic water overuse during the drought requires a critical investigation into Urban Drool’s role in water supply, balance and quality.

The final product will be a map overlay on top of OCPW’s catchment delineations of each monitored storm drain outfall. It will be an Urban Drool Tool that will provide flow data alongside water inefficiency data for the drainage catchments. This Urban Drool Tool will inform watershed management decisions, help employ targeted water conservation strategies, and be instrumental in addressing unnatural water balance in Orange County.

Background:

In 2013, the San Diego Regional Water Board adopted a regional municipal separate storm sewer system (MS4) permit that requires development of a water quality improvement plan (WQIP) for each watershed management area (WMA) within its jurisdiction. The area of south Orange County (south OC) is designated as one WMA comprised of several watersheds (Aliso Creek, San Juan Creek, San Mateo Creek, Dana Point Coastal Streams, and San Clemente Coastal Streams). The municipalities of south OC are currently leading the development of a WQIP for the WMA through a public, stakeholder-driven process. The complete WQIP is due to be submitted to the San Diego Regional Water Board for review and approval by April 1, 2017. Once approved, the WQIP for south OC will identify high priority water quality conditions (HPWQCs), establish short and long term numeric water quality goals, identify strategies to meet those goals, and set schedules as well.

One of the HPWQCs that has been identified for the south OC WMA is Unnatural Water Balance and Flow Regime. The proposed strategies to address this HPWQC focus on identifying and eliminating unnatural and unpermitted, non-exempted dry weather flows from the MS4 into inland receiving waters, with priority for the locations where unnatural dry weather flow inputs arising from an unnatural urban water balance are exacerbating in-stream water quality conditions and contributing to unnatural in-stream regimes. Proposed strategies to address these flows include source control, incentives, and educational measures to promote water conservation and reduction of unnatural flows into the MS4 and structural BMP retrofit strategies to divert and capture water at high priority outfalls, where appropriate.

Over the summer of 2016, OC Public Works performed a comprehensive dry weather flow analysis of approximately 60 stormdrain outfalls that had been identified through a dry weather outfall monitoring program that includes an inventory of 400+ stormdrain outfalls, as having persistent flow. Continuous flow monitoring was conducted at these 60 outfalls over two week time periods and then a composite scoring methodology was applied to prioritize them based on the flow, land use, and other data.

A challenge in addressing this HPWQC will be to know where to target implementation of the proposed strategies on a catchment scale. The purpose of this proof of concept project submittal is to begin to explore water use data from water supply agencies in south OC can help fill data gaps and inform where to most effectively apply strategies.

Procedure:

As proxies for overwatering, we would use storm drain outfall flow data from OC Public Works (OCPW) and water usage and water budget data from Moulton Niguel Water District (MNWD).

Flow from storm drain outfalls is a combination of water from Urban Drool, groundwater seepage, and local permitted discharges. Water usage accounts for both indoor and outdoor water use. MNWD also has a database of calculated water budgets for every one of their customers. It takes into account where water ends up and it budgets for the ideal: no Urban Drool.

Moulton Niguel Water District has calculated a water budget for each household they serve https://www.mnwd.com/understandingwaterbudget/

Subtracting water budget from water use gives the amount of inefficient use of water. This inefficient use could be indoor or outdoor, however, Urban Drool is concerned with only outdoor water use that makes it into the storm drain.

To estimate how much of that inefficient use is Urban Drool and how much Urban Drool contributes to the flow in the outfalls, we could create a ratio of the two proxies: inefficient water use in the drainage catchment vs. flow from outfalls.

Using the census tract bound data from MNWD, and the drainage catchment boundaries of OCPW, the project will calculate a weighted average of water inefficiency (water usage - water budget) for every drainage catchment that we have dry weather flow data for. However, we suspect that the ratio of two week totals would enable us to prioritize education efforts.

The key end product would be the Urban Drool Tool that would showcase a relationship between storm drain outfall discharge and water inefficiency. That relationship can lead us to different management decisions. Where there are high flows during dry weather and high water inefficient use, we would investigate enhanced public water conservation education in that area. If there are high flows at an outfall but low water inefficient use in that catchment, it may make more sense to divert water than increase public education.

Technical Hurdles:

When using inter-agency data, there is often a disconnect between siloed databases. Incongruence between databases can include differences in spatial and time granularity, abbreviations, map projections, units, and software. An additional hurdle that many data analysis projects face is a gap in necessary data; the available data often does not give a complete view of the pathways in and out of a watershed system. When attempting to model a watershed, we may see that the storm drain system is not detailed enough or the monitoring data is not expansive enough. To see where every drop of water travels is not possible with data availability and collection restrictions. Therefore, monitoring the environment in interdisciplinary ways takes time in determining database discrepancies and building available work-arounds. Sometimes well defined meta data can help to patch these differences, but other times the data gaps cannot be filled or the data needs to be manipulated further.

When using the OCPW and the MNWD databases, many of these hurdles needed to be addressed. The MNWD data set is mapped spatially by census tract while the outfall flow data from OCPW is mapped spatially by drainage catchments. Therefore, the project would need to calculate, using the data from MNWD, the spatially weighted average water inefficiency numbers for each drainage catchment that OCPW has delineated.

Fitting the data from MNWD into the OCPW structure works well for our project in question because water quality and flow data downstream is better connected to the OCPW distribution of data. However, in the future the integration may make more sense going the opposite way, depending on the questions asked and the data available. Further Studies:

Currently, the scope of the tool is in identifying areas of high water use inefficiency and overwatering. By integrating with the OCPW water quality database, locations could be further prioritized based on possible contributions to degraded water quality conditions. For example, if we identify a water body with elevated nitrogen or phosphorus content, further investigation could point to over-waterers who are also over-applying fertilizer.

Another future usage of the tool is integrating the Watershed Infiltration and Hydromodification Management Plan (WIHMP) and OverwateringIsOut.org databases of BMP and LID projects. By comparing with the MNWD and OCPW data sets, we can visualize not just if they are helping to improve water quality, but also if we are prioritizing the right location to capture and treat Urban Drool and if our public education programs are effective.

As smart meters become more prevalent among water districts, we expect to see an increase in data granularity. Currently, the water usage efficiency data is reported on a monthly basis while OCPW outfall flow monitoring is performed at two week intervals per site. Therefore, to integrate these two data sets, we assume that flow trends at the outfall remain steady even extending past the two week monitoring period. Increased time granularity provided by smart water meters would allow for more accurate relationships to be drawn and pinpoint exact times of each month where overwatering is more prevalent. A future implementation of this tool would be more accurate at directing water conservation and education efforts.

As with any large data project, we designed a pilot project with the MNWD to determine feasibility and to catalog procedures and common data discrepancies between public works stormwater data and water district data. Our hope is to expand this proof of concept to additional water districts to create a helpful and powerful tool for all of Orange County’s water management. References U.S Environmental Protection Agency. Outdoor Water Use in the United States. Retrieved from https://www3.epa.gov/watersense/pubs/outdoor.html San Diego Regional Water Quality Control Board. Water Quality Improvement Plans. Retrieved from http://www.waterboards.ca.gov/sandiego/water_issues/programs/stormwater/wqip.shtml Orange County Water Quality Improvement Plan, B.3 Chapter https://ocgov.box.com/s/u15i2gp306kqhxgjieyg7y5zqgkffkpy

ca-stormwater-data-challenge's People

Contributors

anudeepvanjavakam1 avatar monobina avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ca-stormwater-data-challenge's Issues

Using Inefficiency Data

Hi all!

Thank you to @monobina for putting together the water inefficiency data. Now the fun part: finding meaning for the public within the pile of numbers. Let's break it down and find how we can give a sentence of data instead of csv's of data to the public:

In the TotalInefficiency.csv:

  • Each line shows total inefficiency in CCF (hundred cubic feet) for one month for a certain census block geoid.
  • The number in that "TotalIEUse" column is all the usage minus the water budget for that month. So the number in the column is total cubic feet over budget. Total over budget consumption is important; it shows how much overwatering is happening (either indoor or outdoor).
  • An example use of this number could be "Last water year (July 2015-2016), your neighborhood used an average of 7,345 gallons of water above what it needed each month. The biggest usages were in April and June."
  • If we use the data like in that sentence, we only need data from July 2015 - June 2016 (The csv goes back to 2011). We can use it in it's current format to find the biggest usage months and we can gather it by water year (@jgrewal has a code for determining water year if we end up calculating this way) to find average monthly overuse.
  • For example, for geoid 60590320022003, we would
  1. gather by the 15-16 water year (Aug 2015 - Jun 2016 because there's no July 2015 data*) and
  2. average the "TotalIEUse" (last column, rows 2 - 12) then
  3. convert to a public friendly unit of gallons.

geoid10

  • The sentence would be "Last water year (July 2015-2016), your neighborhood used an average of 9,657 gallons of water above what it needed each month. The biggest usages were in August and December and the lowest usages were in September and February".

Too High Numbers:

Picturing 9,657 gallons of water per month is a little difficult though. So maybe we only use the above data for highest and lowest usage months and we use the WeightedAverageEfficiency to get efficiency in a proportion instead of in gallons:

In the WeightedAverageEfficiency.csv:

  • Each line shows a weighted average of the ratios of individual water use:budget for one month for a certain census block geoid.
  • For example, if one individual in census block #125 used 3 ccfs and had a budget of 2 ccfs (150% usage:budget) and another individual in census block #125 used 7 ccfs and had a budget of 16 ccfs (<50% usage:budget), the weighted average proportion would be: [(3^2)/2 + (7^2)/2] / (3 +7) = 75%.
  • An example use of this number could be "Your neighborhood typically uses 75% of its water budget"

End Product:

With both these csvs, I see us getting an end product like this: "Your neighborhood typically uses 97% of its water budget. The biggest usages in 2015-2016 water year were in August and January. The lowest usages were in September and February."

@datwater @monobina Please correct me if I'm pulling incorrect meaning from the data. What are some other sentences we could pull from this data too/instead?

Thank you!

Update on Data Needs

An update for Issue #2:

1. x miles from the beach - @leighphan could we calculate this using mapbox? @monobina had said it would be easy to calculate using the geoid in GIS.

2. xxxx landscaped area - this link has irr_area_sf (Irrigated Area of the Household). We should use the average for water year 2015-2016. @monobina when is this data from? The last census?

3. xxx people - this link has hhsize (Household Size). We should use the average for water year 2015-2016. @monobina when is this data from? The last census?

4. xxx water saving improvements - @monobina any updates for #'s 4 - 8?
5. total savings of $XXX
6. competition details
7. Your address is eligible for the XXX rebate program
8. Here’s your goal: xxxxxx xxx xxxx

9. Text and photos regarding stormdrain education (ex: Remember, your stormdrains drain excess outdoor watering directly to the ocean, this outfall drains your neighborhood water into the nearest stream…see photos here...) - @amandaaprahamian will work with OCPW to determine the best verbage for here.

New (see Issue #6)

  • 10. Last water year (July 2015-2016), your neighborhood used an average of XXXX gallons of water above what it needed each month. - use TotalInefficiency.csv to average monthly inefficiency. Decide whether we want this and if yes, address first comment in Issue #6
  • 11. Last water year (July 2015-2016), your neighborhood's biggest usages were in April and June. The smallest usages were in XXXX and XXXX. - find in TotalInefficiency.csv
  • 12. Your neighborhood typically uses XX% of its water budget. - find in WeightedAverageEfficiency.csv

Conveying Water Use Data

Great job so far on the Urban Drool Tool!

Attached is a rough mock-up of what the layout of the Urban Drool Tool could look like. However the layout turns out, the basic data to convey is
(1) spatially: where the census block boundaries, outfalls, and outfall drainage areas are and
(2) with text: what is going on (water use, competitions, savings, and outfall info) in the census block the user is in.

urbandrooltool

Updated Mock Up

Attached is an updated mock-up of what the layout of the Urban Drool Tool could look like.

urbandrooltoolexternal 1

See issue #9 to help edit the dynamic pop-up text that accompanies the map.
See issue #5 for original mock-up and link to Codelab's Urban Drool Tool.

Add questions/concerns/comments to this issue below.

Additional data to be added to census blocks

Currently, only usage data has been joined to the census block shape files.

Join the additional data to the shape files:

  1. x miles from the beach
  2. xxxx landscaped area
  3. xxx people
  4. xxx water saving improvements*
  5. total savings of $XXX*
  6. competition details**
  7. Your address is eligible for the XXX rebate program**
  8. Here’s your goal: xxxxxx xxx xxxx**
  9. Text and photos regarding stormdrain education*** (ex: Remember, your stormdrains drain excess outdoor watering directly to the ocean, this outfall drains your neighborhood water into the nearest stream…see photos here...)

*what is our time frame for these numbers? in the last year? the last 5 years? I suggest a time period that gives us impressive numbers, but is easy to take in (so maybe not 5 long years)?
**applicable only if unique to census blocks. if water district wide, we can add text about it to the web app without needing it joined to the shape files
***some census blocks have no outfalls in view. we need to decide what to write in this section for those census blocks (we could just make it general and have stock photos of outfalls for these census blocks).

Use Data and Dynamic Text in Application

Hi all,

@monobina worked with the water use data and has provided a clean dataset ("IEUsage_Rebate2") and data definitions document ("Data dictionary_Stormwater") for us here.

Please edit the Urban Drool Tool text on box.com as you wish. This will be the dynamic text we use in the application that changes depending on the census block highlighted. (The changing text is highlighted and commented on with which data column will be called on)
(Specifically, if anyone has the ratio of gallons: watering an average yard for a day, we need that for the text.)

There are some columns that have "NA" in them. We will make subsequent scripts for each of those cases and they will be posted here.

In the meantime, these two items can be used to further develop the draft application until final edits are made to the text.

I've closed a lot of the data issues now that we have our finalized data; they can be seen again if you unfilter your view to include closed issues as well.

Use CADataChallengeMap.R for Public Facing Web App basemap

The base map is census blocks with water use data from MNWD associated to each block. There is an overlay of the OCPW drainage catchment boundaries and outfall points.

As of right now, the only data associated with the census blocks is water use data from MNWD. Additional data will be added (see issue "Additional data to be added to census blocks") but hopefully web development can work with this basemap and create the basic functionality of the proposed web app:

The public facing web app will allow a user to type in their address and be directed to a specific census block on this base map. Some addresses will not have catchment boundaries or outfalls in view. A portion of the user's map view would relay the data associated with that census block in sentence form, such as:

  • You are x miles from the beach
  • In your census block, there is xxxx landscaped area and xxx people
  • Your neighborhood is part of xxxxxx competition (details here)
  • Your neighbors have implemented xxx water saving improvements, saving a total of $XXX
  • Your address is eligible for the xxxxxx rebate program
  • Here’s your goal: xxxxxx xxx xxxx
  • Remember, your stormdrains drain excess outdoor watering directly to the ocean, the outfall shown on the map drains your neighborhood water into the xxxxxx stream…

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.