Giter Club home page Giter Club logo

streetlights's Introduction

Stories in Ready

Streetlight Data

Join the chat at https://gitter.im/streetlights-kc/Lobby

This project is intended to make information about how streetlights across the greater Kansas City metro area are used more transparent and accessible.The Streetlight Data project is an effort to create an aggregated database and map of streetlights and information about how they are used.

Want to know about the use cases for our project? please read our KCStreetlightAPI document

Want to know what our latest needs are? Check out our notes from Feb. 19, 2018.

WHY COLLECT DATA ON STREETLIGHTS?

Smart streetlights are at the front lines of the smart city; the switch from traditional high pressure sodium street lights to LEDs is a proven cost saver for cities that can help to finance smart infrastructure. During this switch, the new light poles become part of the Internet of Things: valuable real estate for hanging cameras, sensors, wireless antennas and more. Streetlights also sit in a public right-of-way, a space that often comes with some precedent of public-private collaboration, meaning that private sector companies and entrepreneurs developing smart city applications can deploy technology on existing infrastructure.

Problem Statement But the data about streetlights right now is not easily accessible on a regional scale, to any of these parties: government, nonprofit, private or resident. Cities don’t know how to value this asset, how to choose which tech goes on which pole or how to negotiate with the private sector. The private sector doesn’t know which poles are available or how best to pilot and deploy their new tools. Additionally, most citizens are unaware of the negotiations underway about this critical public infrastructure.

Stakeholders

The direct stakeholder of the Streetlights project is Code for KC/KC Digital Drive, though all participating cities will become stakeholders. Eventually local government, entrepreneurs, researchers, private technology companies and engaged citizens will become direct stakeholders as well.

Cities so far that have agreed to share data are KCMO, Lee’s Summit, and Gladstone. KCP&L has also shared data on polls they own.

Mid-America Regional Council (MARC) is a regional data intermediary who is cooperating in the project as well.

Roadmap/Approach

Data discovery: (IN PROGRESS)

We are working with cities and other streetlight owners to understand first what data they currently collect, how it is stored, and what is available to share. We have collected data in csv form from Lees Summit, KCMO and KCP&L and still asking other cities for data.

Data integration: (IN PROGRESS)

We are building a back-end database to store data from multiple city data, preserve privacy and security where necessary, and develop a database maintenance process. We are in progress of creating a master csv file for this data.

Data visualization: (IN PLANNING)

We are creating a front-end portal to make these data useful to cities, partners, planners, and residents alike. We are still in progress of deciding which language to use and wireframing our project.

How to Contribute / Call to Action

We are looking for:

  • MEAN stack developers to work on the API (right now populating from Google spreadsheet) visualization
  • Database experts to help determine alternatives to Google spreadsheet integration
  • Javascript developers visualizaiton
  • GIS specialists who can read shape files visualization
  • UX and designers as we start to build a front end for different user groups visualization
  • Reporters/analysts to work on how to draw meaning from the data and add additional datasets discovery

The Streetlights project meets every Monday for Hack Nights at Plexpod Crossroads from 6-8 pm.

Team uses Waffle.io for project management Project communication on the #streetlights Slack channel If you are interested, reach out to Aaron Deacon at [email protected] or join our slack.

Developer Notes

What we need to do still:

  1. Translate data to be the same (schema?)
  2. Loading the pole data into the (json) API
  3. Loading the metadata into the api
  4. API support and design work
  5. UI for the API

Three focus areas (December 4)

  1. Create plan for backend (??)
  2. Lee’s Summit and KCMO have two different data formats: Northing and Easting location for poles and long/lat location for poles. We need to unify it in one common format (preferred: latitude and longitude)
  3. Examine KCP&L Shapefile: create master list with KCP&L, Lees Summit and KCMO data
  4. Front End: We need a front end developer to look at it and agree to write in our language
  5. Back End: We need a back end developer to look at it and agree to write in our language

Other Project Info

Test

TODO:

ETL Process

  • Northing & Easting conversion Database (Mongo)
  • Setup methods to enable CRUD operations
  • Create data types for all the data API (ExpressJS)
  • Retrieve data from the database
  • Filter data based on parameters
  • Serve filtered data to the front-end via JSON Front-End (Angular)
  • Display data as a table/spreadsheet (including filtering options)
  • Display data on a map (including filtering options). May use cluster approach

Test

streetlights's People

Contributors

adskjohn avatar adsklowenthal avatar almogbaku avatar amoshaviv avatar andrija-hers avatar anwarjones avatar brownman avatar chanakade avatar chickenham avatar dvorad avatar ellman avatar enkodellc avatar fyockm avatar j9t avatar jackhsu978 avatar jonantel avatar jtanzi avatar jwebbdev avatar khambrickkcdigital avatar liorkesos avatar lirantal avatar matthewscholefield avatar pratik60 avatar rivkabenshalom avatar sarareisner avatar stonelasley avatar timelf123 avatar udisun avatar vikramtiwari avatar zwhitchcox avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

streetlights's Issues

Data normalization script

Script to handle data from multiple sources and format it into the data stream format used by the Streetlight back-end. (Note: Please correct this description if it is not accurate.)

Determine initial data set for launch

We need a clear scope of the data set we will be working with at launch and a sense of how many records we expect to eventually have in total.

Based on what was discussed at our meeting on 2/19/2018, we should probably expect to have approximately 2 million records. This will play an important role as we build out the front- and back-end systems in order to ensure the smoothest user experience possible (within reason) and reducing the number of API calls and database queries required to serve up the data.

Angular App Needs The Following Disclaimer

"This document is provided to the recipient pursuant to Section 610.011, RSMo, which requires public records to be made open to the public for inspection and copying. It has not been prepared for use or reliance by any person or entity other than the City of Lee's Summit. Information and data are supplied by various sources, and the City has made no independent investigation to determine the completeness and accuracy of information or data provided by persons or entities other than the City. The City makes no warranties and/or representations of any kind, express or implied, including without limitation any regarding merchantability or fitness for a particular purpose, regarding information, data, or service provided. All information, data, and service are provided "as is," "with all faults," and by accepting receipt of this document, the recipient accepts the risk of any errors or omissions. In no event shall the City be liable in any way to the users of information, data or service provided by the City. Users of this information, data or service shall hold the City harmless in all matters and accounts arising from the use or reliance on the completeness or accuracy of this information, data or service.
If you would like to contribute to the accuracy of this document, please report any errors or omissions to City of Lee's Summit Information Technology Services. "

Define Variable Names for Data Dictionary

We started a data dictionary for our data. It would be helpful to get variable name descriptions for all of them, so if you know what they are please define them.

Create a api view (frontend)

Create a page that documents and shows the app's API

EDIT: This may need to be scrapped on the frontend. May be turned into a internal/private service.

To do with data

Lee's Summit has Northing and Easting geolocation, which has caused problems. Now we are switching the KCMO geolocation to the Lee's Summit format to see if it has fewer errors.

Split Out And Improve README's

The readme files are a bit verbose, and would be better off being separated.

Things to do:

  • Remove the Team Information(let the issues speak to who's doing what)
  • Keep action Items In issues, and not on the readme's
  • Take advantage of help wanted labels github uses, to let other people join in who may not come to the meetup's but want to work on the project
  • Create a basic arch diagram so new people can quickly understand the project (using something like https://www.draw.io/ or https://www.planttext.com/)

Document the API data models and schema

Document the data models (categories and properties) for all data models that will be handled by the Streetlights API. Also, all methods that the API will support for clients (front-end application) requesting data.

e.g.

Streetlight: {
id: string,
bulbType: 'Inc' | 'LED' | 'Fireflies',
wireless: boolean,
fixture_mfg: string
}

Methods:
getStreetlight(id):
Returns a single streetlight.
Parameters:
id: the ID of the streetlight to be returned.
Returns: JSON

Create API For Mongo Back End Storage

Create REST API for collecting data being posted, and ui requests

{ dataset:"X"
possition:{
lat:56.67
long:-86.57
},
attributes{
{
name:""
value:""
},
{
name:""
value:""
}
}
}

Call Public Works of Raytown

For Street Light data: mainly, to find
location
if it is an LED streetlight
if it has any technology attached to it yet
who owns it

List of good/bad/attention needed data

Identify whether exists, what format its in (file type and lat/long or address)

KCMO:.xls (contains address) and .csv (lat long but in one field so need to break up)

Lees Summit: .shp (

Convert KCMO additions to expected model

We acquired some spreadsheets on 172 streetlights that were switched to LED in September in the Westbottoms. I want to add that to the master list of streetlights

UML Diagram

Will create a UML diagram to better explain layout of project

Call Public Works of Mission, KS

For Street Light data: mainly, to find
location
if it is an LED streetlight
if it has any technology attached to it yet
who owns it

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.