Giter Club home page Giter Club logo

pyopenstates's Introduction

pyopenstates

A Python client for the Open States API v3.

Source: https://github.com/openstates/pyopenstates/

Documentation: https://openstates.github.io/pyopenstates/

Issues: https://github.com/openstates/pyopenstates/issues

Note: This library was recently updated to support Open States API v3, documentation & coverage is a bit behind, but we wanted to get a release out. Feel free to contribute issues and/or fixes.

PyPI badge Test Python

Features

  • Compatible with Python 3.9+
  • Automatic conversion of string dates and timestamps to datetime objects.
  • Tested releases.
  • Set API Key via environment variable or in code.

API Keys

To use the Open States API you must obtain an API Key.

Once you have your key you can use it with this library by setting the OPENSTATES_API_KEY environment variable or calling pyopenstates.set_api_key.

Running tests for this library requires a valid API token

About Open States

Open States strives to improve civic engagement at the state level by providing data and tools regarding state legislatures. We aim to serve members of the public, activist groups, journalists, and researchers with better data on what is happening in their state capital, and to provide tools to reduce barriers to participation and increase engagement.

Open States aggregates legislative information from all 50 states, Washington, D.C., and Puerto Rico. This information is then standardized, cleaned, and published to the public via OpenStates.org, a powerful API, and bulk downloads. OpenStates.org enables individuals to find out who represents them, look up information on an important bill that’s been in the news, discover how their representatives are voting, or just stay current with what is happening in their state. Additionally, our API and bulk downloads see millions of hits every month from advocacy organizations, journalists, researchers, and many others.

Legislative data is collected from official sources, linked at the bottom of relevant pages. In general bill & vote data is collected multiple times a day via our scrapers while legislator data is curated by our team & volunteers like you.

pyopenstates's People

Contributors

alexander-poon avatar bjmc avatar jamesturk avatar johnseekins avatar seanthegeek avatar ssempervirens avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pyopenstates's Issues

sync parameters to API v3

API v3 parameters differ from API v1, and there are still places that this could be fixed in the code

  • bill search
  • bill get
  • metadata
  • get_organizations
  • search_legislators
  • get_legislator
  • locate_legislators
  • search_districts

the v3 docs are pretty good & 100% complete by virtue of being autogenerated, https://v3.openstates.org/ bringing the library in line with those would be a great task for someone interesting in getting a bit more involved

Unit tests hitting API limit despite sleep

Two of the pyopenstates unit tests are failing do to API rate limits, despite a one second sleep between tests. Any ideas, @jamesturk?

======================================================================
ERROR: testBillSearchSort (__main__.Test)
Sorting bill search results
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_pyopenstates.py", line 157, in testBillSearchSort
    per_page=100)
  File "/home/travis/build/openstates/pyopenstates/pyopenstates.py", line 250, in search_bills
    new_results = _get(uri, params=kwargs)
  File "/home/travis/build/openstates/pyopenstates/pyopenstates.py", line 105, in _get
    raise APIError(response.text)
pyopenstates.APIError: {"note": "https://openstates.org/api/register/ for API key. [email protected] to raise limits", "error": "exhausted tokens: 2 req/sec, burst 5"}
======================================================================
ERROR: testPaginatedResults (__main__.Test)
Paginated results
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_pyopenstates.py", line 148, in testPaginatedResults
    results = pyopenstates.search_bills(state="dc", per_page=200)
  File "/home/travis/build/openstates/pyopenstates/pyopenstates.py", line 250, in search_bills
    new_results = _get(uri, params=kwargs)
  File "/home/travis/build/openstates/pyopenstates/pyopenstates.py", line 105, in _get
    raise APIError(response.text)
pyopenstates.APIError: {"note": "https://openstates.org/api/register/ for API key. [email protected] to raise limits", "error": "exhausted tokens: 2 req/sec, burst 5"}

Geo Lookup in API returning errors

Example URL (add your api key): https://openstates.org/api/v1/legislators/geo/?lat=41.9813742&long=-87.6914902&apikey=

The same error is returned for any address using the Geo Lookup in the API v1. It's also been reported in the Discourse here: https://discourse.openstates.org/t/status-code-500-error-as-of-25-aug-2018/114

Error Message:

'Piston/0.2.3rc1 (Django 1.9.13) crash report:\n\nTraceback (most recent call last):\n\n File "/home/openstates/virt/local/lib/python2.7/site-packages/newrelic/hooks/component_piston.py", line 33, in call\n return self._nr_wrapped(*args, **kwargs)\n\n File "/home/openstates/virte51090ea9e0b4384ccb574fcaac07778b0f75996/src/billy/billy/web/api/handlers.py", line 88, in new_read\n obj = old_read(*args, **kwargs)\n\n File "/home/openstates/virte51090ea9e0b4384ccb574fcaac07778b0f75996/src/billy/billy/web/api/handlers.py", line 396, in read\n resp = json.load(urllib2.urlopen(url, timeout=0.5))\n\n File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen\n return opener.open(url, data, timeout)\n\n File "/home/openstates/virt/local/lib/python2.7/site-packages/newrelic/hooks/external_urllib2.py", line 31, in nr_wrapper_opener_director_open\n return wrapped(*args, **kwargs)\n\n File "/usr/lib/python2.7/urllib2.py", line 429, in open\n response = self._open(req, data)\n\n File "/usr/lib/python2.7/urllib2.py", line 447, in _open\n '_open', req)\n\n File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain\n result = func(*args)\n\n File "/usr/lib/python2.7/urllib2.py", line 1228, in http_open\n return self.do_open(httplib.HTTPConnection, req)\n\n File "/usr/lib/python2.7/urllib2.py", line 1201, in do_open\n r = h.getresponse(buffering=True)\n\n File "/home/openstates/virt/local/lib/python2.7/site-packages/raven/breadcrumbs.py", line 375, in getresponse\n rv = real_getresponse(self, *args, **kwargs)\n\n File "/home/openstates/virt/local/lib/python2.7/site-packages/newrelic/hooks/external_httplib.py", line 74, in httplib_getresponse_wrapper\n response = wrapped(*args, **kwargs)\n\n File "/usr/lib/python2.7/httplib.py", line 1136, in getresponse\n response.begin()\n\n File "/usr/lib/python2.7/httplib.py", line 453, in begin\n version, status, reason = self._read_status()\n\n File "/usr/lib/python2.7/httplib.py", line 417, in _read_status\n raise BadStatusLine(line)\n\nBadStatusLine: ''\n" '

pyopenstates.APIError: Piston/0.2.3rc1 (Django 1.9.13) crash report

Hello pyopenstates team,

Thank you for maintaining this great service. I ran into the following issue while looking for state legislators in NY state for a list of schools.

Any idea what could be the problem? My method is pretty simple and actually sleeps for about half a second to overcome and throttling errors. I am only using pyopenstates.locate_legislators()

Thank you,
George Zoto

from time import sleep

def open_states_api(lat_lng_dict):
    sleep(0.4)
    try:
        if lat_lng_dict and ('lat' in lat_lng_dict) and ('lng' in lat_lng_dict):
            my_fields = ['active', 'chamber', 'district', 'full_name', 'sources', 'email']
            legislators = pyopenstates.locate_legislators(lat=lat_lng_dict['lat'], long=lat_lng_dict['lng'], fields=my_fields)
        
            if legislators:
                if len(legislators) == 2:
                    return (legislators[0]['chamber'], legislators[0]['district'], legislators[0]['full_name'], legislators[0]['email'], legislators[1]['chamber'], legislators[1]['district'], legislators[1]['full_name'], legislators[1]['email'])
                elif len(legislators) == 1:
                    return (legislators[0]['chamber'], legislators[0]['district'], legislators[0]['full_name'], legislators[0]['email'], '', '', '', '')
                
        return ('', '', '', '', '', '', '', '')
        
    except:
        print(traceback.format_exc())
Traceback (most recent call last):
  File "<ipython-input-172-8f0b54898bab>", line 8, in open_states_api
    legislators = pyopenstates.locate_legislators(lat=lat_lng_dict['lat'], long=lat_lng_dict['lng'], fields=my_fields)
  File "anaconda/lib/python3.6/site-packages/pyopenstates.py", line 316, in locate_legislators
    return _get("/legislators/geo/", params=dict(lat=lat, long=long, fields=fields))
  File "anaconda/lib/python3.6/site-packages/pyopenstates.py", line 102, in _get
    raise APIError(response.text)
    
pyopenstates.APIError: "Piston/0.2.3rc1 (Django 1.9.13) crash report:

Traceback (most recent call last):

  File \"/home/openstates/virt/local/lib/python2.7/site-packages/newrelic/hooks/component_piston.py\", line 33, in __call__
    return self._nr_wrapped(*args, **kwargs)

  File \"/home/openstates/virte51090ea9e0b4384ccb574fcaac07778b0f75996/src/billy/billy/web/api/handlers.py\", line 88, in new_read
    obj = old_read(*args, **kwargs)

  File \"/home/openstates/virte51090ea9e0b4384ccb574fcaac07778b0f75996/src/billy/billy/web/api/handlers.py\", line 396, in read
    resp = json.load(urllib2.urlopen(url, timeout=0.5))

  File \"/usr/lib/python2.7/urllib2.py\", line 154, in urlopen
    return opener.open(url, data, timeout)

  File \"/home/openstates/virt/local/lib/python2.7/site-packages/newrelic/hooks/external_urllib2.py\", line 31, in _nr_wrapper_opener_director_open_
    return wrapped(*args, **kwargs)

  File \"/usr/lib/python2.7/urllib2.py\", line 429, in open
    response = self._open(req, data)

  File \"/usr/lib/python2.7/urllib2.py\", line 447, in _open
    '_open', req)

  File \"/usr/lib/python2.7/urllib2.py\", line 407, in _call_chain
    result = func(*args)

  File \"/usr/lib/python2.7/urllib2.py\", line 1228, in http_open
    return self.do_open(httplib.HTTPConnection, req)

  File \"/usr/lib/python2.7/urllib2.py\", line 1198, in do_open
    raise URLError(err)

URLError: <urlopen error timed out>
"

Missing legislator data from python wrapper even if web service returns data

Hello pyopenstates team,

I appreciate your hard work. I recently ran into an issue with:
pyopenstates.locate_legislators(lat=lat_lng_dict['lat'], long=lat_lng_dict['lng'], fields=my_fields)

Specifically, I was trying to find the upper and lower chamber representatives for several public schools in Ohio. I realized that for many sites (using their lat/long) pyopenstates.locate_locate_legislators returned no results.

One sample lat long (I can prove about 500 more) is:
{'lat': 39.8748, 'lng': -83.185516}

However, the open states web service at: https://openstates.org/oh/ is able to return results.

Another issue I noticed is that there are 2 results returned for a given chamber and in reality only one of them is currently its representative.

missing data

Do you have any ideas of:

  1. Why pyopenstates.locate_legislators is not able to return an expected result?
  2. Why does the open states return 2 representatives for a given chamber?

I am more than happy to support troubleshooting this case together.

Thank you,
George Zoto

update for API v3

A reminder to myself, should be a fairly light lift since the methods are very similar in function.

Consider migrating from Travis CI to GitHub Actions

Travis CI was bought by Idera some time ago, and immediately laid off half of its engineers -- my understanding is that Idera is a private equity firm that is well known for acquiring companies and essentially stripping them for parts. Travis isn't really being developed anymore.

I would suggest github actions as an alternative, which has better performance and better integration with github.

If you're okay with this @jamesturk, I'm happy to open a PR with this change.

API Key Problem

I am working with the pyopenstates package and I am having a hard time setting up the apikey. When I attempt to use this function pyopenstates.set_api_key(apikey) I get an error saying “‘str’ object is not callable” and when I try to set the environment variable OPENSTATES_API_KEY I still get an error saying that there is no API key. Can you help?

I using a anacanda python 2.7 set up with a jupyter notebook. I was assuming I could just use this set_api_key([my_apikey]).

APIError: {"detail":"exceeded limit

I have tried this

import pyopenstates
import os
os.environ['OPENSTATES_API_KEY'] = "a4....."
results = pyopenstates.search_bills(state="ny", q="taxi")

but consistently got this error

APIError: {"detail":"exceeded limit of 10/min: 11}

without ever getting anything in return from the API. Tried at different time intervals. Could you please advise?

--- EDIT ---

When using the set_api_key functions I get results.

import pyopenstates
pyopenstates.set_api_key("a4...")
results = pyopenstates.search_bills(state="ny", q="taxi")
In [6]: len(results)
Out[6]: 179

Is this an issue maybe related to outdated the documentation?

Encoding used in Legislature data structure

Hello pyopenstates team,

First off, amazing effort and great job on putting this platform together. I am a new user of your api and I only had a question or suggestion for improvement.

I am looking at the Legislature data structure and specifically the encoding for first, last and full name:
http://docs.openstates.org/en/latest/api/legislators.html#legislator-fields

I do not see any mention of encoding used in these fields. My challenge is with representatives that have accents in their names, for example: https://openstates.org/ca/legislators/CAL000057/kevin-de-leon
This could be the case for other fields too that I have not come across yet.

Instead of users trying to guess what encoding you are using, is it possible to:

  1. Update the documentation with this missing piece of information
  2. Suggest a clean way to remove accents from your output

Thank you,
George Zoto

Error when making API calls

When I try to use the pyopenstates API, I get an API error: "error": "quota exceeded: 0/day", "note": "Please consider moving to API v2 or v3. API v1 will be removed in 2021."

My pyopenstates version is up to date with version 1.2.0

It looks like pyopenstates has not been updated since 2018, so I just want to confirm that this error is entirely because the API has not been updated, and that pyopenstates is not usable until the package is updated to the v2 or v3 API. Issue #13 mentions a potential update to v3 it'd also be great to be able to confirm whether this package is under active development, or will not be maintained going forward.

Thanks!

documentation could use another pass

the docs are autogenerated right now, but there are some old references to parameters/etc. that don't exist anymore that were part of v1, if someone is interested in helping out this would be a great place to start! (comment here and I can help guide the effort too)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.