Currently, Access-Control-Allow-Origin in the response has the origin with the trailing slash, which is not a valid origin format, and may never match the one the browser put in the request, and thus arise a blocked by cors policy in the browser.
Ref: https://fetch.spec.whatwg.org/#http-access-control-allow-origin

One of the operational plugins (#41) .

• Endpoint /operations/configuration

Supported operations:

• Report the configuration in use by the application, starting with the startup configuration declared in ba-st/ApplicationStarter

Examples

GET /operations/configuration 

[
  {
    "type": "optional",
    "name": "port",
    "current-value": "8000",
    "default": "4000"
  },
  {
    "type": "mandatory",
    "name": "baseUrl",
    "current-value": "https://api.example.com"
  },
  {
    "type": "flag",
    "name": "debug-mode",
    "current-value": true
  }
]

Move to Hyperspace

• HTTPClientError
• Change HTTPClientError handling allowing instances. Delete HTTPNotAcceptable should be feasible.
• Move ZnEtag and ZnLink
• Move extensions to String #asEtag and #asMediaType
• Move UUID #neoJsonOn: extension
• Move ZnEntity #json: extension
• Move ZnMimeType class #accepts: #asMediaType #quality and #version: extensions
• Move ZnRequest #setUIfMatchTo: and setIfNotMatchTo: extensions
• Move ZnResponse #addLink: #entityTag #links #setEntityTag: extensions
• Move ZnUrl #queryAt:putUrl: and #start:limit: extensions

Explain better how to start the API. See #85

According to MDN:
The response must not contain a body and must include the headers that would have been sent in an equivalent 200 OK response: Cache-Control, Content-Location, Date, ETag, Expires, and Vary.

Provide an abstract subclass of LaunchpadApplication that can be used as a base for applications implementing an API.
It must provide a template for:

• Configuring global error handlers in the API
• Standard configuration required by Stargate
• Ease the creation of RESTful and JSON RPC controllers
• Ease the configuration and creation of the operational plugins with a good set of defaults

The motivation of this feature is to allow API implementors to have access to the negotiated language so they can adapt the responses if needed.

Normative Reference

Accept-Language

The Accept-Language request HTTP header advertises which languages the client is able to understand, and which locale variant is preferred. (By languages, we mean natural languages, such as English, and not programming languages.) Using content negotiation, the server then selects one of the proposals, uses it, and informs the client of its choice with the Content-Language response header.

This header is a hint to be used when the server has no way of determining the language via another way, like a specific URL, that is controlled by an explicit user decision. It is recommended that the server never overrides an explicit decision.

If the server cannot serve any matching language, it can theoretically send back a 406 (Not Acceptable) error code. But, for a better user experience, this is rarely done and more common way is to ignore the Accept-Language header in this case.

Syntax

Accept-Language: <language>
Accept-Language: <locale>
Accept-Language: *

// Multiple types, weighted with the quality value syntax:
Accept-Language: fr-CH, fr;q=0.9, en;q=0.8, de;q=0.7, *;q=0.5

Directives

• <language>: A language expressed as a 2 or 3-character string.
• <locale> A full language tag. In addition to the language itself, it may contain additional information after a -. The most common extra information is the country variant (like 'en-US') or the type of alphabet to use (like sr-Lat).
• Any language; '*' is used as a wildcard.
• q= (q-factor weighting) Any value placed in an order of preference expressed using a relative quality value called weight.

Examples

Accept-Language: de
Accept-Language: de-CH
Accept-Language: en-US,en;q=0.5

Specifications

Content-Language

The Content-Language entity header is used to describe the language(s) intended for the audience, so that it allows a user to differentiate according to the users' own preferred language.

For example, if "Content-Language: de-DE"
is set, it says that the document is intended for German language speakers (however, it doesn't indicate the document is written in German. For example, it might be written in English as part of a language course for German speakers).

If no Content-Language is specified, the default is that the content is intended for all language audiences. Multiple language tags are also possible, as well as applying the Content-Language header to various media types and not only to textual documents.

Syntax

Content-Language: de-DE
Content-Language: en-US
Content-Language: de-DE, en-CA

Directives

• language-tag Multiple language tags are separated by comma. Each language tag is a sequence of one or more case-insensitive subtags, each separated by a hyphen character ("-"). In most cases, a language tag consists of a primary language subtag that identifies a broad family of related languages (e.g., "en" = English), which is optionally followed by a series of subtags that refine or narrow that language's range (e.g., "en-CA" = the variety of English as communicated in Canada).

Note: Language tags are formaly defined in RFC 5646, which rely on the ISO 639 standard

Examples

The Content-Language header is used to specify the intended audience of the page, and can indicate that this is more than one language.

Content-Language: de, en

Specifications

Provide some common abstractions that can be used to implement operational plugins.
An operational plugin must have the following characteristics:

• Must expose operational info and can allow to control some operation
• It's optional. It can be enabled by default but should be possible to disable them
• Must expose at least an endpoint to access it's functionality under /operations/{{plugin-endpoint}}
• Must be secured with a proper authorization filter. In specific situations a plugin can respond with some basic data in case there is no authorization (for example a basic healthcheck) but this behavior must be the exception to the rule.
• #75 Should be possible to enable/disable/configure it on the fly using an API endpoint (given the proper authorization credentials) using the media controls provided in the plugin representation/

Plugin ideas:

• Health Check: #40
• Configuration #42
• Application Info #43
• Loggers #44
• Application Control #45
• Metrics #46

List available plugins

Available plugins including it's status (enabled/disable) and a list of media controls can be accessed doing a GET request to /operations with a supported media type.
Example:

GET /operations
Accepts: application/json

{
  "application-name" : "demo-app",
  "links": {
    "health": { "self": "/operations/health", "status": "ENABLED"},
    "info": {"self": "/operations/info", "status": "DISABLED"},
    ...
  }
}  

When using createEntityTagHashingEncodedResource as the ETag calculation strategy and from:within:get:thenUpdateWith: to implement a route and the controller support several media types; if the client requesting the update doesn't send an Accept header with the same media type used as Content-Type the response will be 412/Precondition Failed even if the provided ETag in the If-Match header is correct.

This happens because the method checking the pre-condition is encoding the resource using the target media type (the one negotiated by the Accept header) instead of using the one in Content-Type.

When fixing this problem we need to take care to not affect the other uses of the ETag calculation (it's used during response generation). The only sender using it not for generating a response is this update method.

In summary, to reproduce the problem all of this needs to happen at the same time:

• createEntityTagHashingEncodedResource is used as ETag calculation strategy
• the controller supports several media types
• the controller implements an update endpoint using from:within:get:thenUpdateWith:
• the client doesn't send an Accept header matching the Content-Type one.

The link [1] in [2] is dead in branch release-candidate.

[1] https://github.com/ba-st/Stargate/blob/release-candidate/docs/how-to/Operations.md
[2] https://github.com/ba-st/Stargate/blob/release-candidate/docs/how-to/how-to-startup-API.md

Now it's required to implement some subclass responsibility methods that are not used if the controller only implements GET operations.

To make it more useful for external objects to access.

Now that cache control is available, it should be mentioned in the corresponding documentation file.

DailyDataAPICommandLineHandler>>basicActivate
Instead of adding runtime error handler straight away, it should be configured in the api and later added during api>>install.
Make sure specific handlers (e.g. for HTTPClientError) are applied before generic handler (e.g. for Error).

Add protocol to the NeoJSON extensions to allow using instance creation methods.
Should also warn when an argument is missing, instead of using nil.

The docs section has a pretty good charting of the architecture of this library, but is there something like a tutorial available? After installing and reading around -- including looking through the tests -- I'm not even sure how to get the pet example running in a live image. Any pointers appreciated!

Fails to load Stargate v2 Test Group. It depends on HTTPNotAcceptableTest (declared in Hyperspace-test) but it is not in the dependency declaration.

Currently, the proper way to configure an error handler to the JRPC request is using JRPCMessageProcessor>>#addErrorHandler:. But that is imposible with actual JsonRPCRequestHandler, and there is no easy way to reuse the ones configured in HTTPBasedRESTfulAPI

ResourceRESTfulControllerTest is useful when testing Controllers, but currently all Stargate tests and examples must be loaded if you need that class, having it on a separate group (Stargate-SUnit-Model ?) would be useful

If an error occurs during the request handling, the after filter does not evaluate. As a consequence of this, the required headers are not added.

handleRequest: aZnRequest ifUnhandled: aBlock
	| response |
	^ [ self evaluateBeforeFilters: aZnRequest.
	     response := self evaluateRouters: aZnRequest ifUnhandled: aBlock.
	     self evaluateAfterFilters: aZnRequest response: response.
	     response	
	   ] on: Exception 
	     do: [ :ex | self exceptionOccurred: ex request: aZnRequest ]

500 Response details are unavailable when configuring CORS correctly. With details I refer to headers and the body.
The browser console says: 'Reason: CORS header 'Access-Control-Allow-Origin' missing'.

Is it possible that the middleware responsible for handling Internal Errors is happening before the CORS one?

Optional Metric for the Metrics Operational Plugin.
We need to collect:

• requests received count
• generated response size
• request/response duration

Labeled by:

• HTTP Method
• URL
• Matched Route (if possible)
• Response status code

This is an umbrella issue to discuss service discovery options.

Once a new version of Teapot is released including:

• zeroflag/Teapot#3
• zeroflag/Teapot#4

we need to:

• update HTTPBasedRESTfulAPI to configure teapot including: {(#notFoundHandlerClass -> Tea405AwareNotFoundHandler)}
• remove our IsUUID class implementation
• fix IsUUIDTest
• remove testMethodNotAllowed as expected failure because now it will work

One of the operational plugins (#41) .

• Endpoint /operations/application-info
• Must be possible to configure this plugin with additional information to report.

Supported operations:

• Report information about the running application

Examples

GET /operations/application-info

[
  "application":{
    "name": "example-api",
    "description": "I'm an example application implementing an API",
    "version": "1.0.0",
    ...
  },
  "pharo": {
    "image": {
      "version": "8.0.0",
      "build-info": "Pharo-8.0.0+build.568.sha.bb9330a322b3cff0cfcf9d76788c3df413422981 (64 Bit)",
      "command-line-parameters": []
    },
    "vm": {
      "location": "/usr/bin/pharo-vm/lib/pharo/5.0-201902062351/pharo",
      "version": "CoInterpreter VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  6 2019
StackToRegisterMappingCogit VMMaker.oscog-eem.2509 uuid: 91e81f64-95de-4914-a960-8f842be3a194 Feb  6 2019
VM: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git Date: Wed Feb 6 15:51:18 2019 CommitHash: a838346b Plugins: 201902062351 https://github.com/OpenSmalltalk/opensmalltalk-vm.git",
      "command-line-options": [],
      "modules": {
        "loaded": [ ... ],
        "built-in": [ "AsynchFilePlugin VMMaker.oscog-eem.2493 (i)", ...]
      },
      "parameters": [ ... ]
    }
  },
  "os": {
    "general": "unix linux-gnu x86_64",
    "details": {
      "description": "Linux Mint 18.3 Sylvia",
      "id": "LinuxMint",
      "release": "18.3",
      "codename": "Sylvia",
      "version": "Linux version 4.15.0-55-generic (buildd@lgw01-amd64-038) (gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)) #60~16.04.2-Ubuntu SMP Thu Jul 4 09:03:09 UTC 2019"
    },
    "enviroment": [
      {"name": "LANG", "value": "en_US.UTF-8"}, 
      ...
    ]
  }
]

One of the operational plugins (#41).

• Endpoint /operations/loggers

Supported operations:

• Report information about the active loggers

Examples

GET /operations/loggers

[
  { 
    "name": "stdout",
    "type": "stdout",
    "links": { 
      "self": "/operations/loggers/stdout"
   }
  },
...
]

Something went wrong with the Tonel writer. Stargate has extensions for these two but the class definitions are still in the directory.

One of the operational plugins (#41) .

• Endpoint /operations/application-control
• Must be possible to configure this plugin with additional methods to control the running app

Supported operations:

• Shutdown: Gracefully shutdown the app

Examples

POST /operations/application-control/shutdown

A new metric with the URL template, not the fully resolved URL would be nice.

It's almost impossible to aggregate them later (it needs to be done case by case which is not maintainable in the long term).

Currently, you get metrics for:
http://example.com/pets/1/favourite-foods/3
http://example.com/pets/1/favourite-foods/2
http://example.com/pets/4/favourite-foods/3
http://example.com/pets/7/favourite-foods?type=fish
I think it would be better if we can provide something like this:
/pets/<pet-uuid>/favourite-foods/<food-uuid>
/pets/<pet-uuid>/favourite-foods

Ideally, it would be a label on top of the current metrics:

http_request_count{url="http://example.com/pets/741/metrics",...,url_template="/pets/741"} 1 1593204634530

But if not possible, a new metric with template_url instead of URL would also be useful.

One of the operational plugins (#41) .

• Endpoint /operations/health-check
• Must be possible to configure this plugin with additional checks to be run.
• In case there are multiple checks the general status will be the worst status of the check runs and every check status can be accessed asking for the details media type and looking at the details object.
• A POST operation with the summary media type can be requested without Authorization, the rest requires an authenticated request with the proper authorization credentials.

Supported operations:

• Run a health check and report the status: POST - application/vnd.stargate.health-check.summary+json
• Run a health check and report a detailed status: POST - application/vnd.stargate.health-check.details+json

Supported statuses

For context see #5

One of the operational plugins (#41) .

• Endpoint /operations/metrics
• Must be possible to configure this plugin with additional metrics to report.

Supported operations:

• Report metrics about the running application

Examples

GET /operations/metrics

To Do: Define the output format, but the info must include:

Virtual Machine Statistics
--------------------------
uptime			0h53m32s
memory			99,753,984 bytes
	old			90,801,952 bytes (91.0%)
	young		4,022,440 bytes (4.0%)
	used		67,200,720 bytes (67.4%)
	free		27,623,672 bytes (27.700000000000003%)
GCs				486 (6610ms between GCs)
	full			1 totalling 87ms (0.0% uptime), avg 87.0ms
	incr		485 totalling 448ms (0.0% uptime), avg 0.9ms
	tenures		45,864 (avg 0 GCs/tenure)
Since last view	1 (1002ms between GCs)
	uptime		1.0s
	full			0 totalling 0ms (0.0% uptime)
	incr		1 totalling 1ms (0.1% uptime), avg 1.0ms
	tenures		0

Update documentation about the new Authentication Filters added during the operational plugins improvements.

Add to the RESTfulRequestHandlerBuilder a configuring message to build the entity has from the encoded resource.

The accept negotiation must only take into account the media types being used to decode objects dropping the ones being used to encode.

Currently, we support 2 strategies for pagination of collections:

• Do not paginate at all
• Page-based pagination

In order to provide more versatility in the pagination strategies a user can use, we can also support:

• KeySet-based pagination
• Cursor-based pagination

A pagination policy now defines:

• Request parameters controlling the pagination
• Hypermedia affordances to provide, both as links or as headers in the response
• Support for using pagination data in query evaluation

Page-based pagination

This is the simplest and most common form of paging. The set is divided into pages.

• Use start and limit query parameters to control pagination
• Provide first, last, next, and previous hypermedia affordances. If needed can provide affordances for jumping to specific pages.

When using SQL in the persistence layer, It can do a query using LIMIT and OFFSET.

Pros:

• You can jump to any particular page
• It allows sending parallel requests with different pages.
• Stateless on the server-side

Cons:

• Bad performance for large OFFSET in SQL.
• It can return repeated or missing data if anything is added/deleted while paginating.

KeySet-based pagination

The API provides a key param that acts as a delimiter of the page. This key param should be the same key of the set sort order. For example, it can be an index (since_id), or timestamp (since_created_at , since_updated_at), etc.

The first request doesn’t contain the delimiter param. The response to this request will contain the value of the key for the last element of the set, so it can be used for subsequent queries.

• Use limit and an API-defined query parameter to control pagination.
• Provide next hypermedia affordance.

When using SQL in the persistence layer, it will end up doing a query including a WHERE and ORDER BY using the key param, and LIMIT for the limit.

Pros:

• The SQL query can be more efficient than using OFFSET
• New records inserted on previous pages won't cause duplicated elements.

Cons:

• It's tied to the sort order.
• There is no way to jump to a specific page. We need to iterate over all the previous pages.
• Missing items if they are added to the previous pages

Cursor-based pagination

Given a set, a cursor will be a piece of data that contains a pointer to an element and the info to get the next/previous elements. The server should return the cursor pointing to the next page in each request. In most cases, the cursor is opaque, so users cannot manipulate it.

The SQL query will depend on the implementation, but it will be similar to the query generated by the KeySet-based Pagination method using a WHERE condition.

Clients should not store the cursor on their side. Google API Documentation suggests adding an expiration date to the token and expiring cursors sent in requests.

• Use limit and after/before to control pagination.
• Provide next and/or previous hypermedia affordances.

Pros:

• If the cursor is opaque, the implementation underneath can change without having to introduce an API change.
• In SQL, for most of the cases, it is much faster than using page since it won’t use OFFSET in the database.
• There is no issue when a record is deleted as opposed to Page-based Pagination

Cons:

• There is no way to skip pages.
• It doesn’t allow sending parallel requests for different batches.
• The implementation is more complex than LIMIT/OFFSET.
• Hard to debug. Given a request, you have to unencode it to see what’s doing.
• Missing items if they are added to the previous pages

Reference

See here some examples of real usage

Future work of #41

Should be possible to enable/disable/configure it on the fly using an API endpoint (given the proper authorization credentials) using the media controls provided in the plugin representation

The current implementation allows to query each plugin status but not change it

This requires the availability of the source code in a deployed image to work.

We need an option that explicitly receives the argument names so it can work without source code access or on platforms that don't have access to the source code un runtime like VAST.

Some APIs will respond with an empty body, and others will respond with a representation of the resource.
Make it configurable.

I've the following method code:

^ ( RouteSpecification
		handling: #GET
		at: self endpoint
		evaluating: [ :httpRequest :context | 'do stuff' ] )
		asCorsAware

Which according to the migration guide should be fixed when loading and executing the Stargate-Deprecated-V6 package. But that package does not contain any rule regarding #asCorsAware and the method is no longer implemented.

Right now, the controllers handle specific exceptions producing the corresponding HTTP error codes, but the library users don't have a clear way to configure it's own exception handlers.
We need to analyze if it's better to use a configurable exception handler for every interaction on the controller, or a more fine grained control.
We also need to consider if it makes sense to configure exception handlers at the API level and not specific controllers.