Giter Club home page Giter Club logo

python-arcgis-rest-query's Introduction

PyPI version travis-ci status

ArcGIS REST Query

A simple library that can download a layer from a map in an ArcGIS web service and convert it to something useful: GeoJSON. If this doesn't suit you , try pyesridump.

Usage

>>> import arcgis
>>> source = "http://services.arcgis.com/P3ePLMYs2RVChkJx/ArcGIS/rest/services/USA_Congressional_Districts/FeatureServer"
>>> service = arcgis.ArcGIS(source)
>>> layer_id = 0
>>> shapes = service.get(layer_id, "STATE_ABBR='IN'")

This assumes you've inspected your ArcGIS services endpoint to know what to look for. ArcGIS DOES publish json files enumerating the endpoints you can query, so autodiscovery could be possible further down the line.

Installation

The easiest way:

pip install arcgis-rest-query

From source:

# Create a virtual environment (pip install virtualenv if you don't have it already)
virtualenv python-arcgis-rest-query
cd python-arcgis-rest-query
. bin/activate
git clone [email protected]:Schwanksta/python-arcgis-rest-query repo
cd repo
pip install -r requirements.txt

From the command line

You can also use the included arcgis-get utility, like so:

$ arcgis-get http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/Legislative/MapServer 0 --where="STATE = 15" > hawaii_congressional_districts.geojson

This will download the 114th Congressional District shapes for Hawaii (FIPS ID is 15). We filter down in this example because there are a bunch of congressional districts, and it would take a while to download them all.

You should run --count_only before downloading an entire dataset, so you can see what you're in store for.

$ arcgis-get http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/Legislative/MapServer 0 --count_only
444

The utilitiy downloads in batches of 1000, so while this will only need to hit the API once, the resulting file would be rather large.

You can also download multiple layers into the same file from the command line. For example, if you wanted to combine the Tennessee congressional districts for the 114th and 113th congress into the same file:

$ arcgis-get http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/Legislative/MapServer 0 12 --where="STATE = 47" --layer_name_field='source_layer' > tn_distrcits_2013_2014.geojson

API

Constructor

The ArcGIS() constructor takes only one required argument, the URL to the web services endpoint you wish to query.

>>> from arcgis import ArcGIS
>>> service = ArcGIS("http://tigerweb.geo.census.gov/arcgis/rest/services/Basemaps/CommunityTIGER/MapServer")

Authenticating requests to your ArcGIS server

If your ArcGIS endpoint is protected via token authorization, pass a valid username/password to the constructor to validate your requests via token auth:

>>> import os
>>> from arcgis import ArcGIS
>>> username = os.getenv('ARCGIS_USERNAME', None)
>>> password = os.getenv('ARCGIS_PASSWORD', None)
>>> service = ArcGIS("http://hostname/to/token/auth/featureServer",
                     username=username,
                     password=password)

You can then continue making requests as detailed below.

ArcGIS.get(layer[,where="1 = 1", fields=[], count_only=False, srid='4326'])

Gets a single layer from the web service.

>>> geojson = service.get(28)
>>> only_florida = service.get(28, where="NAME = 'Florida'")
>>> # Specifying the fields means we get only those fields in return
>>> only_florida_shape = service.get(28, where="NAME = 'Florida'", fields=['OBJECTID'])

If count_only is specified, we return a simple count of the number of features in the layer you're querying. This is useful for determining how big of a query you're about to make, or if your WHERE filter is correct.

>>> states_count = service.get(28, count_only=True)
56
>>> southeast_count = service.get(28, where="NAME IN ('Florida', 'Georgia', 'Alabama', 'South Carolina')", count_only=True)
4

ArcGIS.getMultiple(layers[, where="1 = 1", fields=[], srid='4326', layer_name_field=None])

Concatenate multiple layers into one geojson. This is useful if you have a map with layers for, say, every year, named foo_2014, foo_2013, foo_2012, etc. Setting layer_name_field adds a field to every returned object specifying which layer it came from.

>>> service = ArcGIS("http://tigerweb.geo.census.gov/arcgis/rest/services/Census2010/Transportation/MapServer")
>>> # Get any primary or secondary roads named after MLK Jr. and combine them.
>>> mlk_roads = service.getMultiple([0,1], where="NAME LIKE '%Martin Luther King%'", layer_name_field="src_layer")
>>> # Inspect the src_layer field in the first returned feature.
>>> mlk_roads.get('features')[0].get('properties').get('src_layer')
u'Primary Roads'

ArcGIS.get_json(layer[, where="1 = 1", fields=[], count_only=False, srid='4326'])

Returns the raw JSON from ArcGIS web services for the layer. This is not GeoJSON.

>>> raw_json = service.get_json(0)

ArcGIS.get_descriptor_for_layer(layer)

Returns the JSON descriptor for the layer. This tells you things like what fields are in the layer, what sort of geometry it contains, etc. The response of this function is cached, so repeated calls to the same layer will not hit the ArcGIS web service.

>>> descriptor = service.get_descriptor(0)

ArcGIS.enumerate_layer_fields(layer)

Returns a list of the field names in the layer. Useful for determining what you want to request in a .get() call.

>>> field_list = service.enumerate_layer_fields(0)

Piping to geojsonio

If you install geojsonio-cli, you can pipe output directly to a viewable map.

npm install -g geojsonio-cli

Then, we could re-do the query on Hawaii's congressional districts:

$ arcgis-get http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/Legislative/MapServer 0 --where="STATE = 15" | geojsonio

And get some glorious mapped output: hawaii

Or, for example, if you want to get the Census' state shape for just Florida and display it on geojson.io, you could do:

arcgis-get --where="NAME = 'Florida'" http://tigerweb.geo.census.gov/arcgis/rest/services/Basemaps/CommunityTIGER/MapServer 28 | geojsonio

florida

Potential pitfalls

Since you can only query in batches of 1,000, and sometimes these are millions of records, these operations could take a long time. Currently there's no status indicator on the CLI, so run --count_only first to see how long you might wait.

python-arcgis-rest-query's People

Contributors

anderser avatar cloudniner avatar palewire avatar schwanksta avatar swingley avatar wroscoe avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-arcgis-rest-query's Issues

TypeError: 'NoneType' object is not iterable

Just downloaded and installed to try the sample, and it's not working as I expect:

Edwards-MacBook-Air:~ emv$ arcgis-get http://tigerweb.geo.census.gov/arcgis/rest/services/TIGERweb/Legislative/MapServer 0 --where="STATE = 15" > hawaii_congressional_districts.geojson
Traceback (most recent call last):
  File "/usr/local/bin/arcgis-get", line 35, in <module>
    print json.dumps(arc.get(args.layer[0], where=args.where, fields=args.fields, count_only=args.count_only))
  File "/usr/local/lib/python2.7/site-packages/arcgis/arcgis.py", line 141, in get
    features += [self.esri_to_geojson(feat, geom_parser) for feat in jsobj.get('features')]
TypeError: 'NoneType' object is not iterable

OBJECTID sometimes won't come through

If I tell it to output OBJECTID + a few fields, it works. If I tell it to output all fields, including OBJECTID, it seems to return everything except OBJECTID.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.