dataoneorg / bookkeeper Goto Github PK

This parameter would be added to the GET `./usages' endpoint, for example:

./bookkeeper/usages?quotaType=portal&status=active

and would allow filtering of the usages by the status field, with possible values being active and archived.

This will be handled by the UsagesResource.listUsages() method.

Deploy containerized versions of

1. Postgres with the bookkeeper schema and
2. The bookkeeper application on the production Kubernetes cluster.

The current bookkeeper jar file is a shaded one with external classes from the dependent jar files. When the client (e.g. Metacat) use the bookkeeper jar, it causes some version conflicts. We need a jar file without those external classes.
Here is a link:
https://blog.sonatype.com/2010/01/how-to-create-two-jars-from-one-project-and-why-you-shouldnt/

@csjx @mbjones

Here are two REST endpoints for quota resources:

• getUsage()
  • /quotas/:name/usage?instanceId=:instanceId
• updateUsage()
  • /quotas/:id/usage?usage=:usage

The collection of usages in the first endpoint is based on ":name",
and in the second is based on ":id"

For consistency, the updateUsage endpoint should be updated to use ":name":
/quotas/:name/usage?quotaId::id&usage=:usage

Finish implementation of getUsage:

• can retrieve usage by quota name and instance id
• non-admin users can retrieve usages for users/groups they are associated with

After discussion, we realized that we need a non-authenticated API endpoint that allows public users to find out whether a given portal's usage (by its instanceId and quotaType) is active. This allows us to conditionally render paid features for the portal.

• Add the GET /usages/status?instanceId=:instanceId&quotaType=:quotaType method to the Swagger API
• Create a UsageStatus model with one status property with values of active or inactive:

{
    "status": ":status"
}

Note: Should the UsageStatus model also include the "object": "usagestatus" field to be consistent with our other models?

• Modify UsagesResource and add the getStatus() method (leave out the @PermitAll annotation) which returns a UsageStatus model in 200 responses, otherwise 404 if not found.
• Write up a UsageTest serialization/deserialization unit test for the model using a usagestatus.json fixture

The bookkeeper k8s configuration includes a persistent disk volume that contains the PostgreSQL database that bookkeeper uses.

Setup a mechanism to perform database maintenance operations such as vacuum and backup. Some options:

• cron script
• autovacuum
• pg_dump and rsync to another VM

Discussing this, usage was confusing because of the usages collection, so rename this totalUsage.

Deploy containerized versions of

1. Postgres with the bookkeeper schema and
2. The bookkeeper application on the development Kubernetes cluster.

We expect to have large numbers of calls to listQuotas(), getQuota(), createUsage(), and updateUsage(). stress test these calls using a tool like JMeter with increasing concurrency, such as 10, 100, 1000, and 10000 concurrent calls.

When quotas are listed (QuotasResource.listQuotas()) or a single quota is retrieved (QuotasResource.retrieve()) CORS headers are not returned, causing problems for JS applications which will not allow the results of the query to be accessed unless the required CORS headers are returned. For example, here is a listQuotas() request for quotas that don't exist, causing a 404 result:

curl -v "https://api.test.dataone.org:30443/bookkeeper/v1/quotas?quotaType=portal&subscriber=foo" \
     -H "Origin: https://avatar.nceas.ucsb.edu" \
     -H "Authorization: Bearer $token"
...
> GET /bookkeeper/v1/quotas?quotaType=portal&subscriber=foo HTTP/1.1
> Host: api.test.dataone.org:30443
> User-Agent: curl/7.54.0
> Accept: */*
> Origin: https://avatar.nceas.ucsb.edu
...
< Server: nginx/1.19.0
< Date: Wed, 26 Aug 2020 17:57:38 GMT
< Content-Type: application/json
< Content-Length: 61
< Connection: keep-alive
<
{ [61 bytes data]
100    61  100    61    0     0     38      0  0:00:01  0:00:01 --:--:--    38
* Connection #0 to host api.test.dataone.org left intact
{
    "code": 404,
    "message": "The requested quotas were not found."
}

The Products and Services team requested that the default portal quota be set to 1 and the default trial period set to 1 month.
See https://docs.google.com/document/d/1LstAp9kSj_nyln5pdIrc4vO2iRZpBqr-z4os12mCPt8/edit

Determine how best to accomplish this. A couple of possibilities brought up by @csjx:

• update the standard set 6 products to update the portal count and trial period
• add new products for a trial period
  • for this approach customers would need subscriptions/quota updated when they purchase

Several REST calls require the role

    @RolesAllowed("CN=urn:node:CN,DC=dataone,DC=org")

for example here

Is this still the desired way to authorize privileged operations such as create quotas? As an alternative, could a list of 'super-admin' users be defined?

Since we don't technically support ongoing subscriptions to services, merge them into orders throughout the code base. Also, change subscribers to owners. Products are now purchased as opposed to subscribed to.

This class implements the REST endpoint for obtaining information about a subscription.

First implement the 'listSubscription()` method.

Also enter the subscription endpoint description and parameters into the SwaggerHub documentation.

@csjx In the usages table it is possible to create multiple entries for the same storage, for example

   id | object  | quotaid |                  instanceid                   | quantity | status
  ----+---------+---------+-----------------------------------------------+----------+--------
  5 | storage |       1 | urn:urn:6B444FC1-6CAF-45A4-BC5B-8BD27A6D9B48  |    22220 | active
  6 | storage |       1 | urn:urn:6B444FC1-6CAF-45A4-BC5B-8BD27A6D9B48  |    22220 | active
  7 | storage |       1 | urn:urn:6B444FC1-6CAF-45A4-BC5B-8BD27A6D9B48  |    22220 | active

The instanceid column is indexed, but does not have a UNIQUE constraint.

Should a UNIQUE constraint be added to this column, or alternatively, should the constraint be multi-column, on 'object,instanceid'?

In QuotasResource.listQuotas() and UsagesResource.listUsages(), errors are manually thrown within a containing
try {} blocks, which are getting caught by the containing block, thereby causing the wrong exception type to be
returned, for example:

try {
   if (x) {
       throw WebApplicationException("x error", Response.Status.FORBIDDEN)
  } 
} catch (Exception e) {
   throw ("y error", Response.Status.INTERNAL_SERVER_ERROR)
}

So, if x is true, 'Response.Status.FORBIDDEN' should be returned, but instead 'Response.Status.INTERNAL_SERVER_ERROR' gets returned.

Maybe the easiest way to fix this is to catch 'WebApplicationException' in the containing block and just re-throw it, or
perhaps instead of catching 'Exception' in the outer block, catching expected errors.

Suggestions?

The initial plan for adding bookkeeper to k8s was to have this configuration:

• bookkeeper is run under the bookkeeper Linux user id
• bookkeeper configmaps, pods, services all run in the k8s bookkeeper namespace
• bookkeeper has it's own persistent volume that the Postgresql database is written to/persisted on

The configuration details are being tested on the k8s development cluster (i.e. docker-dev-ucsb-1.test.dataone.org). The dev cluster has one NFS share that is available and that is currently being used by the dev version of metadig-engine. All metadig-engine pods use the persistent volume (pv) and persistent volume claim (pvc) that accesses this share.

One option in this config is to share the pvc between pods in the metadig namespace and the bookkeeper namespace, but it appears that k8s doesn't support this. The supported config seems to be:

• one NFS share
• one pv
• one pvc assigned to a namespace

Another option is to have an additional NFS share that can be used by bookkeper, so set these additional NFS shares:

• the development cluster (docker-dev-ucsb-1.test.dataone.org - /data2 ?)
• the production cluster (docker-ucsb-4.dataone.org - /data2 ?)

It's possible to create a quota db entry with a null subscription, if the subscription attribute is omitted from the JSON object sent via the POST /quotas (QuotasResource.create()) REST endpoint.

SInce quotas.subscriptionId in the quotas table is a foreign key to the subscriptions table, this field should not be null.

So,

• update the Java annotations in Quota.java so that valid quota objects have a subscription id
• add a 'NOT NULL' clause to the quotas table creation clause for subscriptionId

In our design document, the delete usage method should return a boolean result. However, it return a blank string.

Create a Swagger API definition for the bookkeeper REST API.

Document the initial setup, configuration, starting, stoping and debugging the bookkeeper service on k8s.

• put linux user and password in Keybase
• put postgresql user and password in Keybase
• document how to build and deploy the bookkeeper Docker image
  • how to setup repository access, e.g. ~/.m2/settings.xml

A user that is not logged in should be able to view a portal metrics tab. In order to do so, they will not be sending a JWT token.

If this is the case, then bookkeeper should be able to process this request. This request would be specifically to check the usage entry for the portal, for example:

.../bookkeeper/v1/usages?quotaType=portal&instanceId=urn:uuid:FD9BA4E1-C942-48DF-A88A-E09EFAEEC405

@csjx The status column in the usages table can be set or modified by either a create() or an update().

Should the setting of the status column be added to the update_quota_usage_on_insert_or_update trigger, so that on update or create status column of thesubscriptions table is checked and status is set accordingly?

Alternatively, all create() calls might always set status to active, and update will never modify the status value.

@taojing2002 please review and let me know if this is correct.

This parameter is needed by clients (for example, metacat) that maintain a local copy of usages, and need an easy way to retrieve usages that have been created by bookkeeper.

In the case of metacat, it maintains a local table that contains usages that metacat has requested bookkeeper create. When metacat requests that a usage is created, it does not know the 'usageId' that bookkeeper will assign for that usage, as it's a database sequence id maintained by bookkeeper. Metacat would have to asynchronously request the created object from bookkeeper after it is created and potentially keep requesting until the object is created and the object is returned. Then metacat would update it's local store with the usage id. With this scheme, some local usage entries may have the usageId, and some may not.

It's much more reliable and provides better consistency for metacat to not record the usageId, but instead to retrieve usages that it needs using the unique combination of quotaId + instanceId for a usage.

Metacat will use this parameter combination to retrieve a usage in order to get the usageId, that it can then use to request that bookkeeper update or delete usages.

So, the quotaId parameter will be added to the /usages endpoint (listUsages() method)

Enable CORS in the Kubernetes ingress that will be serving bookkeeper request.

https://stackoverflow.com/questions/25775364/enabling-cors-in-dropwizard-not-working#25801822

See NCEAS/metadig-engine#255

As suggested by @taojing2002, the URL parameter name 'subscribers' will be changed to 'subscriber'. The argument for this parameter will be a single subscriber subject. This parameter is available list either quotas or usages.
If it is desired to retrieve usages or quotas for multiple subscribers in a single call, the parameter can be repeated, for example (on my local test server):

locahost:8080/bookkeeper/v1/usages?quotaType=portal&subscriber=http://orcid.org/0000-0002-2192-403X&subscriber=CN=SASAP,DC=dataone,DC=org&subscriber=http://orcid.org/0000-0003-1501-0861"

which will return portal usages associated with the three subscribers specified. Here is the result:

{
    "usages": [
        {
            "id": 1,
            "instanceId": "urn:urn:0001-0001",
            "nodeId": "urn:node:mnOPC",
            "object": "usage",
            "quantity": 1.0,
            "quotaId": 4,
            "status": "active"
        },
        {
            "id": 2,
            "instanceId": "urn:urn:0002-0001",
            "nodeId": "urn:node:mnSASAP",
            "object": "usage",
            "quantity": 1.0,
            "quotaId": 6,
            "status": "active"
        },
        {
            "id": 3,
            "instanceId": "urn:urn:0003-0001",
            "nodeId": "urn:node:mnPCS",
            "object": "usage",
            "quantity": 1.0,
            "quotaId": 8,
            "status": "active"
        }
    ]
}

The update_quota_usage_on_insert_or_update() trigger function that is set on the usage table appears to set the quantity column value incorrectly. The value that is being set for the associated row in the quota table is set to the sum of all values in the usage table.

bookkeeper=> select * from quotas;
 id | object |        name        |   softlimit   |    hardlimit    | usage  |  unit  | subscriptionid |               subject
----+--------+--------------------+---------------+-----------------+--------+--------+----------------+--------------------------------------
  4 | quota  | portal             |             2 |               5 |      5 | portal |              3 | CN=SASAP,DC=dataone,DC=org
  3 | quota  | portal             |            30 |              30 |     15 | portal |              1 | http://orcid.org/0000-0002-2192-403X
  1 | quota  | storage            |    1073741824 |    1181116006.4 | 100000 | byte   |              1 | http://orcid.org/0000-0002-2192-403X
  5 | quota  | portal             |            30 |              40 |     10 | portal |              2 | http://orcid.org/0000-0003-4703-1974
  2 | quota  | repository_storage | 1099511627776 | 1209462790553.6 |        | byte   |              1 | http://orcid.org/0000-0002-2192-403X
(5 rows)

bookkeeper=> select * from usages;
 id | object  | quotaid |                  instanceid                   | quantity | status
----+---------+---------+-----------------------------------------------+----------+--------
  4 | storage |       1 | urn:node:sbpcs                                |   100000 | active
  1 | portal  |       5 | urn:uuid:56925d4b-9e46-49ec-96ea-38dc9ed0a64c |       10 | active
  3 | portal  |       4 | urn:uuid:e2ddeed1-eabe-4a85-b9e9-719a602f0b1e |        5 | active
  2 | portal  |       3 | urn:uuid:FB9FBE56-BC7D-4E2E-A468-397536D31744 |       15 | active
(4 rows)

... now the update

bookkeeper=> update usages set quantity=6 where id=3;
UPDATE 1
bookkeeper=> select * from quotas;
 id | object |        name        |   softlimit   |    hardlimit    | usage  |  unit  | subscriptionid |               subject
----+--------+--------------------+---------------+-----------------+--------+--------+----------------+--------------------------------------
  3 | quota  | portal             |            30 |              30 |     15 | portal |              1 | http://orcid.org/0000-0002-2192-403X
  1 | quota  | storage            |    1073741824 |    1181116006.4 | 100000 | byte   |              1 | http://orcid.org/0000-0002-2192-403X
  5 | quota  | portal             |            30 |              40 |     10 | portal |              2 | http://orcid.org/0000-0003-4703-1974
  4 | quota  | portal             |             2 |               5 | 100031 | portal |              3 | CN=SASAP,DC=dataone,DC=org
  2 | quota  | repository_storage | 1099511627776 | 1209462790553.6 |        | byte   |              1 | http://orcid.org/0000-0002-2192-403X
(5 rows)

bookkeeper=> select * from usages;
 id | object  | quotaid |                  instanceid                   | quantity | status
----+---------+---------+-----------------------------------------------+----------+--------
  4 | storage |       1 | urn:node:sbpcs                                |   100000 | active
  1 | portal  |       5 | urn:uuid:56925d4b-9e46-49ec-96ea-38dc9ed0a64c |       10 | active
  2 | portal  |       3 | urn:uuid:FB9FBE56-BC7D-4E2E-A468-397536D31744 |       15 | active
  3 | portal  |       4 | urn:uuid:e2ddeed1-eabe-4a85-b9e9-719a602f0b1e |        6 | active

Use the io.fabric8 maven plugin to build and push a bookkeeper Docker image to docker hub.
The d1bookkeeper (existing repo) or the dataoneorg (needs to be created) can be used.
If we use the dataoneorg username/repo, this needs to be associated with an email
address. What email address could be used for this?

I ran this command:

curl -k  -v  -H "Authorization: Bearer ${token}" "https://docker-dev-ucsb-1.test.dataone.org:30443/bookkeeper/v1/quotas?subscriber=foo&quotaType=portal&requestor=foo"

It should return not found status or null quota list. However it returned:

{"quotas":[{"id":3,"object":"quota","quotaType":"portal","softLimit":1.0,"hardLimit":1.0,"unit":"portal"},{"id":6,"object":"quota","quotaType":"portal","softLimit":5.0,"hardLimit":5.0,"usage":0.0,"unit":"portal","subscriptionId":3,"subject":"https://orcid.org/0000-0002-1678-0975"},{"id":4,"object":"quota","quotaType":"portal","softLimit":5.0,"hardLimit":5.0,"usage":0.0,"unit":"portal","subscriptionId":1,"subject":"CN=opc,DC=dataone,DC=org"},{"id":10,"object":"quota","quotaType":"portal","softLimit":5.0,"hardLimit":5.0,"usage":5.0,"unit":"portal","subscriptionId":7,"subject":"CN=Robert Nahf A579,O=Google,C=US,DC=cilogon,DC=org"}]}

From @laurenwalker

"The product object that is returned in a Subscription doesn’t have an obvious attribute to differentiate portal products from hosted repo products. They all have type: service , so the name attribute seems to be the only way to tell apart different product types. But the names don’t look to be strictly controlled, for example, Small Organization . It would be nice to have a product category attribute (or similar), where category would be set to one of a defined list of product types, such as portals, hostedrepo, etc."

I'd like to switch the k8s configuration to use NGINX virtual server routing instead of the ingress resource that we have been using for metadig-engine. This feature has been available since NGINX v1.5.0 (May 2019)

NGINX will still be used as the ingress controller, which will use the virtual server and virtual server route resource to route requests from the main k8s URL to the services running in k8s.

NGINX has stated that it will continue to support ingress resources, but will halt any new development or features for them. All new development is being put into the virtual server and route mechanism.

The components needed to support this are:

• the NGINX ingress controller
  • the manifest files (.yml) will continue to be in the metadig-engine repo
  • the controller will serve both metadig-engine and bookkeeper services
• the virtual server
  • the manifest files for this will be in the metadig-engine repo, under 'nceas-nginx'
• the virtual server route
  • the manifest files for this will be in the bookkeeper repo
  • metadig-engine will have a similar route .yml in the metadig-engine repo

Here is the current REST API for listQuotas:

QuotaList = listQuotas():   GET     /quotas?\
                                            subscriber=:subscriber&\
                                            quotaType=:quotaType&\
                                            requestor=:requestor

and listUsages():

UsageList = listUsage():    GET     /usage?quotaType=:quotaType\
                                            &instanceId=:instanceId&\
                                            subscriber=:subscriber

Is the requestor parameter needed for listUsages()?

Calling PUT /usages/<id> produces this error:

{"code":417,"message":"Couldn't update the usage: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint \"usages_quotaid_instanceid_idx\"\n  Detail: Key (quotaid, instanceid)=(17, urn:urn:9abc-defj) already exists.

Here is the script making the request:

curl -v -H "Authorization: Bearer ${token}" -X PUT \
     --data @${data} \
     -H "Content-Type: application/json" \
     -H "Accept: application/json"  \
     "http://localhost:8080/bookkeeper/v1/usages/55"

... and the data:

{
   "id":55,
   "object":"usage",
   "quotaId":17,
   "instanceId":"urn:uuid:9abc-defj",
   "quantity":2.0,
   "status":"archived",
   "nodeId":"urn:node:SASAP"
}

We need to know which repository is related to a given Usage of a Quota:

• Change the model and add a nodeId attribute.
• Change the quotas table and add a uniqueness constraint for the subscriptionId and quotaName field combinations so we only have one row for a given quota type associated with a given subscription.
• Change the Quota.name field to Quota.quotaType and reflect it in the schema, classes, and tests.

During multiple developer meetings, it was determined that the quotas and usages REST endpoints should be updated to conform to this list:

QuotaList = listQuotas():   GET     /quotas?\
                                            subscriber=:subscriber&\
                                            quotaType=:quotaType&\
                                            requestor=:requestor
Quota     = createQuota():  POST    /quotas
Quota     = getQuota():     GET     /quotas/:id
Quota     = updateQuota():  PUT     /quotas/:id
boolean   = deleteQuota():  DELETE  /quotas/:id
Usage     = getUsage():     GET     /usage/:id
UsageList = listUsage():    GET     /usage?quotaType=:quotaType\
                                            &instanceId=:instanceId&\
                                            subscriber=:subscriber
Usage     = createUsage():  POST    /usage
Usage     = updateUsage():  PUT     /usage/:id
boolean   = deleteUsage():  DELETE  /usage/:id

This will require a new UsagesResource class that will handle the /usage endpoints. The usage methods will be removed from QuotasResource. The new class/method configuration will be:

• QuotasResource:
  • listQuotas()
  • createQuota()
  • getQuota()
  • updateQuota()
  • deleteQuota()
• UsagesResource:
  • listUsage()
  • createUsage()
  • getUsage()
  • updateUsage()
  • deleteUsage()

Bookkeeper quota usages currently can be in one of these states (aka 'status'): "active" or "archived".

Change "archived" to "inactive", to avoid confusion with the use of the term "archived" in DataONE.

Make this update in all source code, database scripts, documentation.

The file ./resources/db.migrations/V1.8__Insert_Standard_Products.sql appears to be out of date, as it referes
to quota types such as branded_portals, repository_storage_quota.

It would be useful to review this entire file to ensure that it is current.

To support the requestore parameter for QuotasResource.listQuotas() and UsagesResource.listUsages, overload DataONEAuthHelper to include a subject parameter.
The current DataONEAuthHelper.getCustomerWIthSubjectInfo() accepts a JWT token, which then extracts the subject from the token and calls the CN accounts service to get subjectInfo.

The overloaded method will call the CN accounts service directly (with the specified subject) so won't require listQuotas, listUsages to pass a token to be parsed.

Add a test to UsageStoreTest to verity that when a usage is added, the trigger function that updates the quota table usage field is working properly.

This parameter would be added to the GET `./quotas' endpoint, for example:

./bookkeeper/quotas?quotaType=portal&status=active

and would allow filtering of the quotas by the status field, with possible values being active and archived.

This will be handled by the QuotasResource.listUsages() method.

The following request should return HTTP 404, but returns 202 instead:

curl -v -X GET "https://api.test.dataone.org:30443/bookkeeper/v1/quotas/10000" \
     -H "Origin: https://avatar.nceas.ucsb.edu" \
     -H "Authorization: Bearer $token"
...
< HTTP/1.1 204 No Content
< Server: nginx/1.19.0
< Date: Tue, 25 Aug 2020 21:29:13 GMT
< Connection: keep-alive
< Access-Control-Allow-Origin: https://avatar.nceas.ucsb.edu
< Access-Control-Allow-Methods: GET, POST, OPTIONS, PUT, DELETE
< Access-Control-Allow-Headers: Authorization, Content-Type, Origin, Cache-Control
< Access-Control-Allow-Credentials: true

If the quota is found, the proper HTTP result is returned.

We manage usages through the UsagesResource (and the QuotasResource), and need to finish the implementation of:

• QuotaList = listQuotas(): GET /quotas?subscribers=:subscribers&quotaType=:quotaType&requestor=:requestor
• Usage = createUsage(): POST /usages
• Usage = updateUsage(): PUT /usages/:id
• boolean = deleteUsage(): DELETE /usages/:id

The administrative listing of usages can come in a later release, but I'll list it here too:
UsageList = listUsage(): GET /usages?quotaType=:quotaType&instanceId=:instanceId&subscribers=:subscribers

Several of the bookkeeper services return Http status 417 (Expectation Failed), which isn't consistent with the error that occurred.

Consider using one of these Http status codes:

• 404 Not Found
• 500 Internal Server Error
• 503 Service Unavailable

Here is a sample request/response:

GET /bookkeeper/v1/quotas?quotaType=portal&subscribers=foo&requestor=http://orcid.org/0000-0002-2192-403X HTTP/1.1

{"code":417,"message":"The requested quotas couldn't be listed: http://orcid.org/0000-0002-2192-403X requested subscribers that don't exist or requestor doesn't have privilege to view them."}

For this case, where there was not an internal error (e.g. no SQL Error), status 404 seems the most appropriate.

In other instances, 417 is returned if any exception is thrown. In these cases, 500 may be more appropriate.

@csjx for an earlier issue, 'quota.name' was changed to 'quota.quotaType'. That would have been a good time
to also update 'subject' to 'subscriber', but that didn't get done. The user facing instances of 'subject' have been changed to 'subscriber'. Should the internal instances of 'subject' be changed?

The curl command:

curl -k  -v  -H "Authorization: Bearer ${token}" "https://api.test.dataone.org:30443/bookkeeper/v1/quotas?subscriber=CN=opc,DC=dataone,DC=org&quotaType=portal"

It returns a quota with usage 2.0

{"quotas":[{"id":4,"object":"quota","quotaType":"portal","softLimit":5.0,"hardLimit":5.0,"usage":2.0,"unit":"portal","subscriptionId":1,"subject":"CN=opc,DC=dataone,DC=org"}]}

However, when I list the usages associated with subscriber. It got two usages. However, one is inactive and it should NOT be counted. So the quota should have one usage rather than two.

curl -k  -v  -H "Authorization: Bearer ${token}" "https://api.test.dataone.org:30443/bookkeeper/v1/usages?subscriber=CN=opc,DC=dataone,DC=org&quotaType=portal"

It return with two usages. But one is inactive:

{"usages":[{"id":60,"object":"usage","quotaId":4,"instanceId":"urn:uuidc11e3c52-dad2-4ddb-8aeb-099e1ffe6533","quantity":1.0,"status":"active","nodeId":"urn:node:METACAT_TEST"},
{"id":61,"object":"usage","quotaId":4,"instanceId":"urn:uuidd991d85a-99ca-46b3-8f04-c8c560083505","quantity":1.0,"status":"inactive","nodeId":"urn:node:METACAT_TEST"}]}

In our design document, the rest call looks like:

UsageList = listUsage():    GET     /usage?quotaType=:quotaType&instanceId=:instanceId&subscriber=:subscriber

So the Jason String should be a UsageList object which contains a list of usage. However, it directly returns a list of usages.

curl -k   -H "Authorization: Bearer ${token}" "https://docker-dev-ucsb-1.test.dataone.org:30443/bookkeeper/v1/usages/?quotaId=4&instanceId=urn%3Auuidfa4df84d-31c0-430c-bb2d-506662cec307"
[{"id":4,"object":"usage","quotaId":4,"instanceId":"urn:uuidfa4df84d-31c0-430c-bb2d-506662cec307","quantity":1.0,"status":"active","nodeId":"urn:node:METACAT_TEST"}]

listQuotas() should support calls where there is no subscriber parameter specified.

If not subscriber is specified, then the list of subscribers that will be used is the set of groups and equivalent identities (all associated subjects) that will be retrieved from the DataONE accounts service.

If the caller wishes to return entries for just their subject, they should specify ?subscriber=<their subject>

Move documentation from https://github.com/csjx/d1-membership-plan-mgmt/blob/master/membership-plan-management.rst to the docs directory in this repo, and modify as needed for the 1.0.0 release.