openreferral / api-specification Goto Github PK
View Code? Open in Web Editor NEWThis is the working repository for Open Referral's Human Services Data API protocols.
Home Page: https://openreferral.readthedocs.io/en/latest/hsda/
License: Other
This is the working repository for Open Referral's Human Services Data API protocols.
Home Page: https://openreferral.readthedocs.io/en/latest/hsda/
License: Other
Proprosal to introduce a custom HSDA media type for bulk loading, which would be a single data point for GET and POST of data.
Open311 has a similar initiative - http://wiki.open311.org/
While still offering an "everthing/" path on main API, this would allow the core API to reflect the flat nature of schema, keeping usable by spreadsheet users.
This is the first proposal to use media types for breaking projects into smaller chunks so they do not affect other types of users -- meeting the needs of systems users, without impact say web, spreadsheet, SPA, or conversational interface users.
While I am putting this under v1.2 -- the bulk API would branch and take on its own versioning, as separate branch.
IMO, search URLs are always tough. Right now, the search URL in the Ohana API I've seen is a bit confusing because I don’t know what kind of resource it will return. Will it return locations
, or organizations
, or any type of resource that fits the keyword? Also, a search
isn’t a resource exactly, so having it in the URL doesn’t feel very RESTy (although arguably this doesn’t matter much). My convention would be to just add those search params to the /locations
endpoint, unless the search action is meant to return multiple resources, in which case I think a dedicated search
URL is appropriate.
On this note, is it necessary to have an endpoint for /locations/nearby
when that URL accepts params anyway? Thoughts on just allowing those possible params on the /locations
endpoint instead of dedicating a separate endpoint for it?
I am working on a JSON schema for HSDS, and for HSDA.
I already have an API for validating an API path or JSON snippet against any JSON schema when ready. Will release as extra set of API paths.
Once schema are establish we will need to establish the set of rules being applied, and how strict we intend on being.
I would like to propose a separate HAL representation be developed, allowing for HSDA access to be done via relationships between the data.
This is the second first proposal to use media types for breaking projects into smaller chunks so they do not affect other types of users -- meeting the needs of systems users, without impact say web, spreadsheet, SPA, or conversational interface users.
Implementors can choose to follow core schema, or be HAL compliant, choosing to go a more hypermedia path for all clients.
While I am putting this under v1.2 -- the hypermedia API would branch and take on its own versioning, as separate branch.
Introducing a fourth dimension of the scope / filtering conversation -- project level. How do parts of the schema and API definition get grouped, and evolved separately from other projects.
How is this separation described in API definition(s) - OpenAPI + APIs.json, which can then be reflected in documentation, SDKs, and other tooling.
This also needs to be thought about as it impacts the definition for a "HSDA compliant API or schema" - what % compliant is an API. Few will ever be 100% compliant, but providers can pick and choose which project areas they support or don't support.
How we are going to handle pagination in the requests and responses for the APIs. This overlaps with the hypermedia discussion. Currently we are using a page, and per_page parameters for GET requests. Now is the time to change or evolve.
I'm opening up a 3rd dimension to the whole filtering conversation. It is relevatng data filtering (#22) and schema filtering (#21), but is at a higher level.
It answers other concerns about the overall API design surface area being too large, too many paths. Path filtering in the API documentation would all for a default set of paths (ie. orgs, locations, services, search), then mid, and advanced tiers.
This would reduce cognitive load for developers when first getting going, but allow for knowledgeable users to get at a wealth of more advanced or precise search endpoints.
I am suggesting this also to handle future addition of API paths, such as utility ones like schema validators, or possible universal ID, taxonomy, messaging, etc. Additional projects, relevant to, and sometimes part of core spec, but off in separate namespace.
Migrating from a Slack channel discussion around the number of available APIs I'd like to open a thread specifically about what the core paths GET and POST structures should look like
When you GET, what can be expected? Simple or complete nested (ie. phone, contacts,)?
When you POST or PUT, what can be expected? Simple or complete nested (ie. phone, contacts,)?
This overlaps with other conversation regarding schema filtering, or how you tell the API which schema the user would like returned - #21
This also overlaps with conversation regarding /search/ and it's relationship to /organization/, /locations/, and /services. Where the results should be - #22
My recommendation is that all four core paths have a simple and complete schema representations that the consumer can choose from when GET, or POST and PUT. You choose whether you want to get a full listing or summary listing -- we can also craft other views in the future.
This conversation is probably one of the most pressing regarding the v1.1 design discussion.
Adding a parameter for filtering the schema.
Providing comma separated list of fields, and sub-resources.
Suggesting we call it fields=
The major part of this release was ensuring there was POST, PUT, and DELETE methods for the entire surface area of the HSDS schema.
right now we use POST for adding, and PUT for updating. Both use the body and PUT updates the entire resource (ie. organization, location, service, contact). I suggest we add PATCH, allowing for partial updates, as well as OPTIONs further defining the surface area.
Had a comment come in on a blog post that I want to consider for future road map when it comes to expanding on the regular and holiday scheduling.
Source: http://apievangelist.com/2017/05/23/avoid-moving-too-fast-for-my-api-audience/#comment-3321395153
Abstract: I am very familiar with 211 from the service provider standpoint. The NAACP in Eugene provides a weekly tutoring session at the local community college for students of color. Every year I have to update our listing on 211. Our schedule ends up changing as the school year progresses and we keep all the events on our wordpress site. I have dreamed of a day when a plug in would arrive so the 211 site would "listen" for changes on the events app on word press. This would make it so I could easily update our 211 listing with the most current info. I am probably not alone and a lot of small daycare providers, tutors, camps and counselors are using wordpress and other out of the box (OOB) websites to manage their events. Can we get some API action from the calendar devs on the OOB websites? (please note I am not a developer, just a user)
I would also recommend holding off on the API for the metadata portion of the schema - https://openreferral.readthedocs.io/en/latest/reference/#metadata
This area needs some additional considering when it comes to managment, logging, and other aspects of API operations.
Provide guidance on HTTP Status codes to be returned. Add all relevant HTTP Status Codes to individual endpoints using the OpenAPI.
The location response is quite big, and on one hand this is awesome, because it ensures the client by default gets the info it needs. However, it’s quite a hefty payload and any mobile clients would prefer something smaller. What about having an option to request this resource with a minimum response (e.g. just the location resource, excluding organization, categories, etc) so that clients concerned about package size can opt for something smaller?
See Irakli's post about one possible approach to this: http://www.freshblurbs.com/blog/2015/06/25/api-representations-prefer.html
what is plan for operationId? camelCase?
We need use cases for people who are going to be using any HSDS/A implementation.
Submit an issue with any format you want about a use case you'd like to see the API serve.
Please provide as much detail as you possibly can.
There are personas in Issue #24 , but I'd like to hear real world scenarios, not just personas.
I have a working JSON schema API that we can pass any JSON file to it and it will validate against a JSON schema.
I need to finish JSON schema representation for:
I am still polishing my JSON schema chops, but also I'm thinking deeply about a mininum viable definition, as well as potentially more verbose definitions.
I'm thinking that we should be allowing for different levels of compliance.
This thinking overlaps with the schema filtering conversaiton for the API -- whether someone wants to see simple or complex responses, or even POST/PUT to the API - Issue 21.
THis would be the time to adopt JSON API, HAL, or other evolutionary approach to responses - let's discuss.
I'd like to introduce the concept of API definition filtering to the HSDA design discussion. In an effort to help ensure the API design speak to as wide as possible audience as we can, I'd like to build on discussion around data filtering, and schema filtering, and allow for labeling and packaging up of the API paths using the OpenAPI tagging.
In the OpenAPI, each path can have one or many tags associated with it. We can create a simple set (/organizations/, /locations/, /services/, /search), and a more complete set with all the direct paths to sub elements--reducing cognitive load for new users, while also allowing for access to precise paths for directly updating phone number, or an address without being bothered with entire schema.
I'll work on an example to show how API definition filtering can be used to reduce complexity in API documentation.
This has been migrated from Ohana (codeforamerica/ohana-api#417 (comment)), and worth considering as part of specification:
As part of Code for Miami's involvement with the Knight Cities Challenge, we recently conducted a Civic User Testing Group in Miami with a reskinned version of the Ohana web search. You can find our thoughts here.
We then built a follow-up prototype and performed CUTGroup testing for that prototype as well.
While developing this follow up project, we noticed the API is not configured to return taxonomy data in a keyword search. To get around this, our app ran an initial keyword search to get category data, iterate through each result, and then a second search for that particular organization’s details.
As a result, if a search for the keyword “food” returns 50 organizations, the app must make 51 calls to the API (one for the initial search, and one for each of the 50 organizations in the results). As a result, the app suffered from noticeable performance issues that would not be sustainable at scale.
We can solve this in two ways:
Refactor the API to return taxonomy data in keyword search results; or
Create a second API that allows third parties to copy the vendor provider’s entire data set into their database, which they can then customize for more efficient searches.
Currently, there is a sample data set here:
https://github.com/openreferral/sample-data
But this only contains a subset of the models in the specification.
We should add the rest of these models so that this data is complete and can be used in prototyping and automated testing.
I didn't quite see any feedback regarding the options on the table for allowing for a more comprehensive approach to schema filtering, so I'm just going to go with the most intuitive option, and add an /everything path to /organizations, /locations, /contacts, and /services.
Ie. when you do /organizations/ you get a simple, flat schema.
when you do /organizations/everything you get a complete schema with all subresources.
We can revisit header options for this, and other approaches further down the road, this should support the concerns on the table.
Opening up a thread dedicated to the approval, feedback, and auditing flow for the platform, that allows anyone to update a record in the system, but there is an approval flow that can be applied before the change is accepted.
This area would overlap with the meta data conversation which involves the change log and logging - #28
This is a project to begin defining a set of objects to be returned as part of error codes, providing a structure, and common language set to describe common, as well as custom error responses.
I want to introduce this thread to overlap with discussions around data filtering #22, schema filtering #21, response structure #6, and hypermedia #7.
Basics of content negotiation would be, do we offer application/csv, and text/html, as well as applicaiton/json (which is current default).
Advanced would be offering application/vnd.api+json, application/hal+json, or application/vnd.siren+json.
I'm thinking that offering a simple CSV and JSON response for default make sense for newbies. Content negotiation 101. Then we could start loading up some of the more detailed requests, and solving the "return everything" and "return exactly what I want" with introduction of hypermedia type, or even possibly something like application/vnd.hsds+api.
Currently ALL POST and PUT endpoints use the body for a transport of the schema.
How will we be filtering API responses? Right now we depend on the /search/ path to do search, with the following fields:
category:
email:
keyword:
language:
lat_lng:
location:
org_name:
radius:
service_area:
status:
How will this be applied across paths?
This is the thread to discuss the versioning plan for the API. The starting suggestion is to use the current v1.0 and v1.1, with keeping in alignment with semantic versioning used by major providers like Google - http://apievangelist.com/2017/03/09/guidance-on-versioning-your-api-from-google/.
The current plan is to place the version in the url, right after the baseurl for the API. It's open for discussion for changing strategy with the current release. After that we will probably lock in.
Let's say I have an organizational phone id (let's say 12345
), and I want to get the whole record from the API. But I don't necessarily know the right organization id.
The format for phones is: /organizations/{orgId}/phones/{phoneId}
Is there a way for me to GET (or PUT changes) without knowing the organization id? Based on my current reading of version 1.1, there isn't. Should we add something like Google that would let me GET /organizations/-/phones/12345
for this situation?
Aside from Documentation, the links at https://openreferral.github.io are turning up 404...
This is a use case as part of the Use Cases meta-issue (#44). It expands on @klambacher 's "Integrators" use case:
Integrators want to combine information into existing systems or datasets. They are committed to the system they use, and may have a substantial existing data set that is outside of the collection policy of the target dataset they want to integrate. They care deeply about ongoing maintenance and updates of the information and often move large quantities of information frequently. Over time, this can sometimes become a bi-directional sharing relationship if the initial integration is successful. Efficient ways to determine changes and keep data in sync between systems is what matters, and search features are generally irrelevant beyond the ability to query changes and do sanity checks. Efficient movement of data in bulk is important and pull-and-display APIs often require too many requests to collect information. "I'd like to integrate your information into our existing database, on this schedule"
Here's what folks are trying to achieve:
How will we be sorting information. There is no sorting capability in the current design.
I want to begin mapping out the webhooks guidance for future versions of the specification. I'd like to find several webhook implementations we could use to model the external push aspects of an implementation, making things more event oriented.
What are the other API design considerations?
I'd like to introduce guidance for messaging. Suggesting it is a standalone layer to the system that other areas use. To suppor the approval, and other aspects of the system we will need messaging.
I'm thinking this will include email, SMS, and potentially Github issues and Slack integration when it makes sense. Possibly even a templating system.
Need a plan for status codes, starting with 200, 404, and 500, but then getting more sophisticated in how APIs should be returning errors.
Review and consider this existing AIRS work alongside specification - https://www.airs.org/i4a/pages/index.cfm?pageID=3363
I wanted to have an issue for continuing discussions around the paths that have been defined so far.
Currently all paths reflect the schema, but are very plain English, and intuitive.
I am using dashes instead of underscores. Except path variables use underscores as defined by the schema.
It's a good start.
This is the thread to discuss use of hypermedia as part of the HSDS v1.x. Now is the time to have this discussion, and discuss the evolution of the project.
I'm suggesting we look at the following formats, based upon their adoption in the API sector:
JSON API - http://jsonapi.org/
Collection+JSON - http://amundsen.com/media-types/collection/
HAL - Hypertext Application Language - http://stateless.co/hal_specification.html
Siren - https://github.com/kevinswiber/siren
I'll be working on a separate entry, outlining the benefits of applying hypermedia design, as well as working examples to help visualize how it will be used.
Do we want to provide domain guidance for implementations? Recommend folks do api.example.com, or example.com/api.
Consider offering portal advice too, like developer.example.com.
I'm starting a read for aggregating conversations at the granular level. The looseness or strictness of how the JSON schema interprets HSDS or HSA.
I do not see them in the schema?
what are the headers, and stragegy for this.
This has been migrated from Ohana (codeforamerica/ohana-api#419 (comment)) - worth revisiting:
JordanLyons commented 23 days ago
Several programs and agencies in my area don't want their physical or mailing addresses possible. I see that this is a required field for several of the CSV's, and I wonder if there's anything that could be done to prevent that kind of sensitive info getting published on the database.
@greggish
greggish commented 22 days ago
This is a known challenge for things like domestic violence shelters, human trafficking services, etc. Before 2016, I would have called it "an important edge case" but now i don't think that 'edge case' does it justice.
The simple scenario seems to be that there's usually a 'front gate' point of contact that is public, without sharing details about where people actually go once they've entered into relationship with the service.
I'm going to cc @kinlane on this (although i suspect that delineating between public and private addresses is not likely to be an issue we can directly address before 2.0). I'm also curious about how other referral providers handle stuff like this, and it might be an issue you could bring to the Google Group for discussion.
I want to open up the conversation around using GraphQL as part the design for the HSDS API. This approach is being considered because HSDS implementations often are data intensive systems, with advanced querying needs. We just want to make sure everything is on the table.
GraphQL - http://graphql.org/
The table name INTEPRETATION_SERVICES is probably meant to be INTERPRETATION_SERVICES in the .yaml file.
Now that I'm digging though, I'm not sure which git repo file is the correct yaml file. This issue relates to the yaml in the documentation at https://openreferral.github.io/api-specification/definition/yaml/
Wanted to have a place to have conversations about how we reflect actions in the API design. Right now the API design is very CRUD, and there really aren't any actions beyond just verbs.
I'd like to have an ongoing discussion around how we reflect more advanced actions in the API design, allowing for more actions to be taken after we get more feedback from the community.
I will be moving the /search option beyond just returning locations, and give it a collection of organizations, locations, services, and contracts.
I will also give /search an /everything, so that we can have a simple, and expanded results.
Need to come up with the standard for returning error and status codes -- there is no plan so far. I'd like to see this span a standard error responses structure, as well as application of HTTP status codes.
HTTP Status Success
HTTP Status Redirection
HTTP Status User Error
HTTP Status Server Error
Opening up a discussion around a universal unique ID system for records, that all vendors can participate it, allowing single records to have a unique IDs across sytems. Need to conduct more research on best practices, and engage in conversation with vendors.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.