citygram / citygram-services Goto Github PK

View Code? Open in Web Editor NEW

16.0 4.0 37.0 297 KB

Web content transformation proxies for open data API's

Home Page: https://citygram-services.herokuapp.com/

Ruby 96.25% HTML 0.37% Dockerfile 0.86% TSQL 2.52%

citygram-services's Introduction

spy-glass (aka Citygram connector)

Spyglass is a Code for America project by the Charlotte Team for the 2014 fellowship.

Why are we doing this?

What does this do now?

This is a registry of micro ETL endpoints. What does that even mean? Good question. Citygram has good overview documentation.

But here's a specific example of the information flow:

CKAN code enforcement -> Citygram connector -> Citygram

This particular Citygram connector pulls the last seven days worth of code complaints and formats them as geojson that Citygram polls many times a day.

It creates a unique key that is a composite of the CaseNo and Status columns. When this key changes, Citygram will create a new event that will be sent to subscribers. Since the key is a composite, an event is created whenever a status changes for a given case.

What will this do in the future?

TODO

Who is this made by?

Setup

Install Ruby

git clone https://github.com/citygram/citygram-services.git
cd citygram-services
cp .env.sample .env
gem install bundler
bundle install
bundle exec rackup

citygram-services's People

Contributors

Stargazers

Watchers

citygram-services's Issues

Seattle 911 Fire Dispatches

Definition: 911 calls to Seattle's Fire Department.
Data Source: https://data.seattle.gov/Public-Safety/Seattle-Real-Time-Fire-911-Calls/kzjm-xkqj
ETL: https://citygram-services.herokuapp.com/seattle-911-fire-calls
Template: Seattle Fire Department was dispatched to [address] at around [datetime]. The event was categorized as "[type]".

SF Street Use Permits

Definition: Street use permits issued by SF Public Works for sidewalk work, excavations, water breaks, etc.
Data Source: https://data.sfgov.org/City-Infrastructure/Street-Use-Permits/b6tj-gt35
ETL: [forthcoming]
Template: [Salutation] A new permit has been issued for [Permit Type], at [streetname] between [Cross Street 1] and [Cross Street 2], from [permit_start_date] to [permit_end_date].

Example: Hello! A new permit has been issued for Excavation, at GEARY BLVD between BAKER ST \ SAINT JOSEPHS AVE and LYON ST, from 10/07/2014 12:00:00 AM to 11/20/2014 12:00:00 AM.

Categories to filter for:
Emergency
Excavation
NightNoise
Sidewalk
TempOccup

SF Temporary Tow Away Zones

Definition: Temporary tow away zones in San Francisco granted by permit from the SFMTA or Department of Public Works.
Data Source: https://data.sfgov.org/Transportation/SFMTA-Enforced-Temporary-Tow-Zones/cqn5-muyy
ETL:
Template: [salutation] There is a temporary tow away zone near [FromAddress] [StreetName] from [starttime]-[endtime] on [Days] from [starting_date] to [ending_date] due to [Category_Code].
Example: Yo! There is a temporary tow away zone near 353 Folsom Street from 7:00-21:00 on Monday-Friday from 8/16/2014 to 10/16/2014 due to Construction - City Contract.
Template V2 if one-day tow zone (logic: if [starting_date] = [ending_date]): [salutation] There is a temporary tow away zone near [FromAddress] [StreetName] from [starttime]-[endtime] on [Days] [starting_date] due to [Category_Code].
Example V2 if one-day tow zone: Yo! There is a temporary tow away zone near 353 Folsom Street from 17:00-21:00 on Thursday 10/16/2014 due to Special Event - Event.

Lexington Foreclosures

Definition: A property that has been seized for failure to make mortgage payments and sold at auction.
Data Source: [forthcoming]
ETL: [forthcoming]
Template: A foreclosed property has been sold near you at [address] for [price].

Seattle Land Use Permits

Definition: An official approval to create or modify individual parcels of property, including unit subdivisions, lot boundary adjustments; may include a public comment process, environmental reviews; and does not authorize construction activities.
Data Source: https://data.seattle.gov/Permitting/Land-Use-Permits/uyyd-8gak
ETL: http://citygram-services.herokuapp.com/seattle-land-use-permits
Template: A [category] land use permit near you at [address] has been filed. It is a [permit type] permit type, and you can find out more at [url].

Miami-Dade Building Permits

I noticed today that our "Building Permits" dataset for Miami only pulls from the Miami-Dade county dataset, which isn't a complete view of building permits. Municipalities like the City of Miami, Miami Beach, et al. have separate building permit datasets.

If we wanted to merge these datasets, how would that work? Would we add more code to https://github.com/citygram/citygram-services/blob/master/lib/spy_glass/registry/miami-building-permits.rb to pull from each municipality in addition to the county, come up with a way to rewrite them to look similar to the existing county entries?

Integration of rubocop?

Hey!

I was wondering what the opinion of integrating Rubocop for style-checking would be. In addition to enforcing a consistent style, it also helps find issues like unused variables, improper equality operators, missing documentation, and other possible bugs.

Figured I'd ask before integrating it since it'll take a little time to address the issues that rubocop cannot autocorrect.

Seattle Code Violations

Definition: Complaints received by Seattle's Department of Planning and Development that have become cases for further investigation.
Data Source: https://data.seattle.gov/Community/Code-Violation-Cases/dk8m-pdjf
ETL: https://citygram-services.herokuapp.com/seattle-code-violation-cases
Template: There's been a [case_group] code violation near you at [address]. Its status is "voluntary compliance", and you can find out more at [url].

Charlotte Historic District

Currently suffering from a NoMethodFound error:

2015-08-18T16:26:20.082408+00:00 app[web.1]: NoMethodError - undefined method `[]' for nil:NilClass:
2015-08-18T16:26:20.082413+00:00 app[web.1]:    /app/lib/spy_glass/registry/clt-historic-district-reviews.rb:13:in `block (2 levels) in

In addition, the title needs to be revisited.

Lexington Building Permits

Definition: An official approval required for new construction or adding onto pre-existing structures.
Data Source: [forthcoming]
ETL: [forthcoming]
Template: A building permit application has been submitted near you at [Address]. The permit is for [PropertyType] and its ID is [PermitID].

Lexington deploy

@bigfleet I forgot I'lll need your services to deploy #37 to Heroku.

Thanks for the help

SF New Business Locations

Definition: New businesses opening soon.
Data Source: https://data.sfgov.org/Economy-and-Community/Open-Business-Locations-San-Francisco/g8m3-pdis
ETL: [forthcoming]
Template: [Salutation] A new business called [DBA Name] is opening soon at [Street_Address].

Example: Hey! A new business called INFINITI SUSHI is opening soon at 4112 GEARY BLVD.

SF New Street Tree Plantings

Definition: When new trees are planted along city streets by the San Francisco Department of Public Works.
Data Source: https://data.sfgov.org/City-Infrastructure/Street-Tree-List/tkzw-k3nq
ETL: [forthcoming]
Template: [Salutation] A new tree has been planted near you by the SF Department of Public Works at [qAddress]! It is a [qSpecies] and has been planted at a [qSiteInfo].
Example: Hi! A new tree has been planted near you by the SF Department of Public Works at 605 Divisadero St! It is a Gingko biloba :: Maidenhair tree and has been planted at a Sidewalk: Curb side : Cutout.

SF Mobile Food Truck Permits

Yay food trucks! https://data.sfgov.org/Permitting/Mobile-Food-Facility-Permit/rqzj-sfat.

@tchu88 can you make a sweet food truck icon?

Tulsa earthquakes have duplicate dates

https://www.citygram.org/digests/6d6aeb8a-e6ac-49e7-a8a3-1384f732219f/events shows duplicate dates for all the earthquakes?

Add a test (suite?) based on GeoJSONLint

This site can take a POST and return whether the feed is valid or not. That's a helpful finding, and can help provide early feedback to those that are writing a Citygram service.

San Diego Code Enforcement

Definition: A way to be notified of code enforcement activity near an address in San Diego.
Data Source:: OpenDSD
ETL:: Data is already being ETLed out to OpenDSD, and we just grab the latest.
Template:: "There's been a code enforcement case updated near you at: [Street Address].
The violation is "[Description]".
You can find out more at: [url with more info]

URL is https://sd-notifications.herokuapp.com/dsd/code-enforcement

Thank you!

citygram.org: bad layout on iPhone

Sorry I couldn't find a repo for the website. ❓

The citygram.org home page is busted on iPhone iOS 9:

[noop]

Setting an alert trigger on this repo.

Fixing bug that doesn't show what address that permits are being applied for

From this CfM comment:

"I sign up to get the permits that are pulled, but all it does it gives the names of the people pulling the permits and that's not necessarily important to me. What I want to track is the address; what property that permit was pulled for and who pulled it."

The JSON field name was changed, so we had to modify our code in Miami building permits

Seattle Electrical Permits

Definition: An official approval required for commercial projects for electrical wiring to be installed, altered, extended, or connected to any electrical equipment.
Data Source: https://data.seattle.gov/Permitting/Electrical-Permits-Current/raim-ay5x
ETL: https://citygram-services.herokuapp.com/seattle-commercial-electrical-permits
Template: Hi! [contractor] has applied for a commercial electrical permit at [address]e. Find out more at [url].

Pending rezonings data issue

Output does not pass GeoJSONLint with a missing geometry issue. This is preventing new updates from being circulated.

Endpoint /clt-pending-rezoning

List of salutations.

Hi!
Hello!
Salutations!
YO!
Hi there.
Hey.
Hola.
Ahoy!
Aloha!
Ciao.
Hi :)
Good day!
Greetings!
Look!

SF Crime Incidents

Definition: Major crime incidents reported by the SFPD, updated daily.
Data Source: https://data.sfgov.org/Public-Safety/SFPD-Incidents-Previous-Three-Months/tmnf-yvry
ETL: [forthcoming]
Template: A crime incident happened near you on [Date] at [Address]. The SFPD described it as a [Descript] with the following resolution: [Resolution].

Example: A crime incident happened near you on 9/29/2014 at FULTON ST / HYDE ST. The SFPD described it as a ATTEMPTED ROBBERY WITH A GUN with the following resolution: NONE.

Categories to filter for:
LARCENY/THEFT
BURGLARY
VEHICLE THEFT
ASSAULT
ROBBERY
ARSON
KIDNAPPING
SEX OFFENSES, FORCIBLE
SEX OFFENSES, NON FORCIBLE
STOLEN PROPERTY

Charlotte Land Development

Definition: The Land Development Division of E&PM is responsible for the review and inspection of residential subdivisions and commercial development projects in order to ensure compliance with various City ordinances and standards.
Data Source: https://aca.accela.com/charlotte/Cap/CapHome.aspx?module=LandDevelopment&TabName=LandDevelopment
ETL: https://citygram-services.herokuapp.com/clt-land-development
Template: [salutation] A Land Development plan for [address] has been submitted. The description is: [description]. Learn more: [url]

Seattle Building Permits

Definition: An official approval required for new construction or adding onto pre-existing structures.
Data Source: https://data.seattle.gov/Permitting/Building-Permits-Current/mags-97de
ETL: https://citygram-services.herokuapp.com/seattle-building-permits
Template: A building permit for *[category] [permit_type] has been submitted near you at [address]. The proposed value is [value]. Find out more at [url].

Lexington Code Enforcement Violations

Definition: An update of the code enforcement database, signifying a new or status updated code complaint.
Data Source: http://104.131.23.252/api/action/datastore_search_sql
ETL: https://citygram-services.herokuapp.com/lexington-code-enforcement-complaints
Template: A code complaint has been opened or updated near you at [address]. Its status is [status] & its case number is [caseno].

SF Current City Advisories

Definition: Current events and advisories that affect San Francisco.
Data Source: https://data.sfgov.org/Culture-and-Recreation/Current-San-Francisco-Advisories/qn4d-bra2
ETL: [forthcoming]
Template: [Salutation] There's a new SF advisory: [TITLE]. For more info, get the SF311 app at sf311.org/mobile or call 415-701-2311.

Example: Hey! There's a new SF advisory: Ebola Fact Sheet. For more info, get the SF311 app at sf311.org/mobile or call 415-701-2311.

Seattle 911 Police Incidents

Definition: All Police responses to 911 calls within the City of Seattle. (To protect the security of a scene, these events are added only after the incident is considered safe to close out.)
Data Source: https://data.seattle.gov/Public-Safety/Seattle-Police-Department-911-Incident-Response/3k2p-39jp
ETL: https://citygram-services.herokuapp.com/seattle-pd-911-incidents
Template: A 911 police incident has occurred near you at [hundred_block_location]. It was described as [event_clearance_description] and has been cleared.