zowe / api-layer Goto Github PK
View Code? Open in Web Editor NEWThe API Mediation Layer provides a single point of access for mainframe service REST APIs.
License: Eclipse Public License 2.0
The API Mediation Layer provides a single point of access for mainframe service REST APIs.
License: Eclipse Public License 2.0
Issue by stevenhorsman-ibm
Tuesday Sep 18, 2018 at 09:09 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/110
An example of this (which blocks moving atlas fully behind the API Gateway):
In the Atlas USS apis we use the uss file path as a path parameter in the URI eg.
http --verify=no -a stevenh:XXXXX GET "https://winmvs3b.hursley.ibm.com:7443/Atlas/api/uss/files/%2Fu%2Fstevenh" HTTP/1.1 200 OK … { "children": [ { "link": "https://winmvs3b.hursley.ibm.com:7443/Atlas/api/uss/files/%2Fu%2Fstevenh%2FnewDirectory123", "name": "newDirectory123", "type": "directory" }, …
I've got the Atlas APIs generally working behind the gateway:
http --verify=no -a stevenh:XXXXXX GET "https://localhost:10010/api/v1/explorer-server/datasets/STEVENH.JCL/members" [ "ANOTHER", "COPY", "FROMDATA" ]
but when I make a request which has the '%2f' in I get a bad request:
http --verify=no -a stevenh:XXXXXX GET "https://localhost:10010/api/v1/explorer-server/uss/files/%2Fu%2Fstevenh" HTTP/1.1 400 Connection: close Content-Length: 0 Date: Wed, 12 Sep 2018 10:43:59 GMT
My guess is that the %2F is causing some issues in the Zuul filter, but I'm far from an expert!
Issue by vojoup
Tuesday Jul 31, 2018 at 08:42 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/14
vojoup included the following code: https://github.com/gizafoundation/api-layer/pull/14/commits
Issue by taban03
Wednesday Aug 01, 2018 at 16:46 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/21
Removal of postinstall
taban03 included the following code: https://github.com/gizafoundation/api-layer/pull/21/commits
Issue by supmi01
Friday Aug 31, 2018 at 08:45 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/49
Architecture and business objective:
Completed research:
Acceptance Criteria:
User stories:
Issue by varma09
Monday Oct 01, 2018 at 07:28 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/137
Business objective:
As an OSG developer, I would like to make API Mediation Layer source code open source in Zowe GitHub repository (https://github.com/zowe) so the community can contribute to it.
Stories:
parent #138
parent #139
parent #140
parent #141
Success criteria:
1.) Code follows Zowe rules for open-sourcing
No closed source dependencies
License text is updated
For open sourcing, the following checklist should be met:
Repository LICENSE file to EPL 2.0
All files license headers to SPDX-EPL 2.0 (See Imperative repo for LICENSE_HEADER file, we used gulp automation to run through this)
No closed source dependencies of any kind or LGPL used in the build – open source only.
Even pre-built binaries pulled in at build time are no-good.
There are LGPL dependencies in discoverable-client (Hibernate)
It can be easily removed
Veracode scan - https://rally1.rallydev.com/slm/attachment/252893449440/2018-09-12_1455.png
Steve Winslow from the OMP will scan the repository for adherence to the above 3 bullets and approve the repository
Close all outstanding branches and PRs
Migrate to Zowe org, WITHOUT HISTORY (Under generic userid). We cannot move the history, as pre-EPL-2.0 code, even in historical format, is a legal issue for the OMP.
We have a script we can share to automate this step (Mark)
Migrate any issues to Zowe repo (automation or manual, nothing official from the group for this)
2.) The code is available https://github.com/zowe/
3.) The Jenkins is using the new repository
4.) The integration tests are executed as part of Zowe Jenkins (can be in Docker) - not a must for open-sourcing
5.) The "com.ca.mfaas" is changed to "org.zowe.apiml" - not a blocker for initial open-sourcing but all other contributors are doing it
Issue by supmi01
Monday Sep 03, 2018 at 19:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/74
Business value:
Sherman (system administrator) struggles with the amount of logged messages produced by the API layer. Each of its components produces a lot of Java-specific trace log, which is useless. Sherman only needs to see the meaningful information that helps him quickly troubleshoot the problems.
As Michelle, I would like to see error messages that can help me understand why my request has failed in the REST API response, so I do not need to ask Sherman to look into the log.
Acceptance criteria:
Amount of system log messages by default is reduced down to only important for system administrator (no debug-like details)
External option to control amount of log information
Documented important log messages
Documented process how to on debug messages in case of investigation of problems
Error message related to the routed requests should be returned to the client and not logged into the server log (to prevent using a lot of JES spool - there is a story for special error log after GA)
Additional stories:
Update Eureka startup to UP only if the service is fully started and functional (e.g. API Gateway is connected to Eureka) so /application/health is UP when the service can be used without problems
Notes:
This should not stretch (bad logging hides important error messages and makes product difficult to develop and maintain)
Issue by supmi01
Monday Sep 03, 2018 at 12:41 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/67
None
Issue by varma09
Monday Oct 01, 2018 at 07:30 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/140
As a Zowe engineer, I would like to have the integration tests executed as part of the Zowe build so I know that my changes have not broken them.
Not a must for open-sourcing
Acceptance Criteria (accepted by the architect):
The integration tests are executed as part of Zowe Jenkins (can be in Docker on Jenkins without z/OS)
Their results and logs are available by clicking on a link in the GitHub PR
Notes:
does it need to be run on mf only? no!
need docker container, run all integration tests, posts results to Git.
run on branch and master? yes
need to check if we can change zowe and if not find out who can do it for us - probably Mark
jenkins plugin for publishing results
need to come up with a procedure for reviewing external pull requests
how to support integration tests/build issues
might Mark/Dan/Petr help with this
Issue by cZikos
Tuesday Jul 24, 2018 at 11:21 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/6
cZikos included the following code: https://github.com/gizafoundation/api-layer/pull/6/commits
Issue by plavjanik
Tuesday Sep 11, 2018 at 14:27 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/100
As Sherman (sysprog) I would like to define z/OSMF service to the APIML so it can be used via the gateway (e.g. from Zowe CLI).
Acceptance criteria:
Add a basic test to zit to check zosmf routing?
Depends on:
Issue by plavjanik
Wednesday Sep 26, 2018 at 13:19 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/128
As a system programmer, I do not want to provide the IP address where the service is running so only the hostname is provided.
Currently, Discovery Service (Eureka) requires the IP address of the service to be provided by the service (and therefore by Sherman - a system programmer persona who installs Zowe or applications that integrate into Zowe) in addition to the hostname to ensure correct access to the service. This is not easy to be determined because the service does not know what is the correct network adapter to detect the correct IP address. The result is that Sherman has to provide it.
The goal is to implement a solution that will not require any entry of the IP address for services.
Acceptance criteria:
Architectural requirements:
Issue by supmi01
Friday Sep 07, 2018 at 14:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/94
Error initializing secureHttpClient: /var/zowe/0.9.0/explorer-server/../scripts/internal/../../api-mediation/scripts/../keystore/api_gateway.ts (EDC5129I No suh file or directory.)
the pb is here (see the line above) : /api-mediation/scripts/../keystore
in my installation filesystem , i have:
EUID=0 /X9/var/zowe/0.9.0/api-mediation/scripts/keystore/
Type Perm Changed-GMT-1 Owner ------Size Filename Row 1
_ Dir 740 2018-09-06 12:28 ROOT 8192 .
_ Dir 755 2018-09-06 12:28 ROOT 8192 ..
_ File 640 2018-09-06 12:28 ROOT 2339 api_gateway.ks
_ File 640 2018-09-06 12:28 ROOT 1048 api_gateway.ts
i created a dir /X9/var/zowe/0.9.0/api-mediation/keystore/ (see it is not not /api-mediation/scripts/keystore/ but api-mediation/keystore/, and i copied the 2 files api_gateway.ks and api_gateway.ts
Issue by dkelosky
Tuesday Sep 25, 2018 at 14:30 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/122
If you launch a sample discover-able application specifying:
title: ${environment.uiTileTitle:SDK Applications}
description: ${environment.uiTileDescription:Sample REST API Applications}
Those two items properly show in the catalog UI:
However, if you alter the mfaas.catalog-ui-title.title
and mfaas.catalog-ui-title.description
and restart the service, catalog UI does to reflect the changed values.
Issue by vojoup
Wednesday Jul 25, 2018 at 09:27 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/10
vojoup included the following code: https://github.com/gizafoundation/api-layer/pull/10/commits
Issue by MarkAckert
Wednesday Jul 11, 2018 at 03:52 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/1
TODO: Convert to multi-branch and credential identifier updates.
Setup jenkins CI/CD pipeline on OSG Infrastructure.
MarkAckert included the following code: https://github.com/gizafoundation/api-layer/pull/1/commits
Issue by supmi01
Monday Sep 03, 2018 at 19:27 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/75
Business value:
A major feature of the API Layer is the dynamic discovery of API services which requires physical changes to API provider code through adding Eureka enabler code piece. However not every API service that would benefit from inclusion into API layer can be updated/changed - this is especially true for customer home-grown API services. The API layer should provide a way to onboard such APIs. This also would apply as a temporary solution for APIs those code change would take long.
parent of #38
parent of #83
parent of #84
parent of #85
parent of #86
parent of #87
parent of #88
parent of #89
parent of #90
parent of #91
parent of #99
parent of #100
parent of #161
Issue by stevenhorsman-ibm
Wednesday Sep 19, 2018 at 14:21 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/113
Some other builds eg atlas-wpl-packaging and zowe-install-packing push artifactory build information as part of their builds. This means that we can reference artifacts based on their build rather than having them hard codeds (eg include "atlas-wlp-package-pipeline :: master").
We should consider updating the API ML build to do the same. The pipeline plugin info might help here: https://www.jfrog.com/confluence/display/RTF/Working+With+Pipeline+Jobs+in+Jenkins if we want to the the jenkinsfile rather than gradle scripts
Issue by varma09
Wednesday Sep 05, 2018 at 13:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/90
As Sherman, I would like API layer to detect changes in the directory automatically (in an interval that I can specify).
Issue by plavjanik
Monday Aug 27, 2018 at 07:00 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/38
This is to address feedback of @vmelnychuk: How to reproduce this feature, so we can add integration tests to not forget to disable this in future? as we have test for https://github.com/gizafoundation/api-layer/blob/master/integration-tests/src/test/java/com/ca/mfaas/gatewayservice/RemoveHeaderDuplicationIntegrationTest.java
Issue by supmi01
Sunday Sep 02, 2018 at 20:28 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/66
Business objective:
As an Zowe developer, I would like to make API Mediation Layer source code open source in Zowe GitHub repository (https://github.com/zowe) so the community can contribute to it.
Success criteria:
Code follows Zowe rules for open-sourcing
No closed source dependencies
License text is updated
For open sourcing, the following checklist should be met:
Repository LICENSE file to EPL 2.0
All files license headers to SPDX-EPL 2.0 (See Imperative repo for LICENSE_HEADER file, we used gulp automation to run through this)
No closed source dependencies of any kind or LGPL used in the build – open source only.
Even pre-built binaries pulled in at build time are no-good.
Steve Winslow from the OMP will scan the repository for adherence to the above 3 bullets and approve the repository
Close all outstanding branches and PRs
Migrate to Zowe org, WITHOUT HISTORY (Under generic userid!). We cannot move the history, as pre-EPL-2.0 code, even in historical format, is a legal issue for the OMP.
We have a script we can share to automate this step (Mark)
Migrate any issues to Zowe repo (automation or manual, nothing official from the group for this)
The integration tests are executed as part of Zowe Jenkins (can be in Docker)
The code is available https://github.com/zowe/
The Jenkins is using new repository
parent #73
Issue by cZikos
Wednesday Aug 01, 2018 at 12:52 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/19
These changes will manage the labels of each pull request, according to the outcome of the build.
Whenever there is a new successful build, it will renew the labels so CA automated pipeline will start testing again.
cZikos included the following code: https://github.com/gizafoundation/api-layer/pull/19/commits
Issue by plavjanik
Wednesday Aug 01, 2018 at 04:35 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/17
Removes debugging and internal informational messages from the default configuration of the logs.
The development logs can be turned on by using dev
Spring Boot profile as described in the internal documentation -
https://github.com/gizafoundation/api-layer/pull/17/files#diff-f8dc3534cf0da751863e6075858d6379R81
The Eureka errors are silent for some time after the services are started so confusing error messages are not displayed immediately and services have time to connect to each other properly.
There are other improvements that can be done in logging. These will be addressed in other pull requests in order to make this PR small.
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/17/commits
Issue by varma09
Wednesday Sep 05, 2018 at 13:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/89
As Sherman, I would like API layer to display service status in the API catalog based on the availability of the APIs and route only to active instances
Notes:
Issue by varma09
Wednesday Sep 05, 2018 at 13:23 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/87
As Sherman I would like to see API documentation of APIs were defined by static definition and provide Swagger:
Issue by supmi01
Wednesday Sep 12, 2018 at 15:32 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/104
None
Issue by plavjanik
Monday Jul 23, 2018 at 20:37 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/4
This change adds support for WebSocket routing in the API layer.
It adds a simple WebSocket handler to discoverable-client for testing purposes, it adds routing of WebSocket connections to gateway-service and integration tests. The documentation about how to add routing into the Eureka metadata is in docs/websockets.md
.
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/4/commits
Issue by vmelnychuk
Wednesday Aug 01, 2018 at 14:04 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/20
Make Gradle scripts more cleaner.
vmelnychuk included the following code: https://github.com/gizafoundation/api-layer/pull/20/commits
Issue by MarkAckert
Tuesday Jul 24, 2018 at 03:49 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/5
MarkAckert included the following code: https://github.com/gizafoundation/api-layer/pull/5/commits
Issue by supmi01
Sunday Sep 02, 2018 at 20:08 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/57
Objective:
As a Michelle, I want to be notified in a nice way if something unexpected happens to the catalog
Acceptance criteria:
Unexpected errors and warnings are displayed in a dialog
Notes:
Should be modal (dependency on Mineral team)
Issue by plavjanik
Thursday Jul 26, 2018 at 18:55 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/11
This updates the basic documentation to contain good information on how to start all the services in a simple way.
The configuration for local machine has been moved Markdown text to YAML files so the commands to start any service is very simple and short and the documentation is connected to a working configuration.
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/11/commits
Issue by plavjanik
Tuesday Jul 24, 2018 at 20:48 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/9
This change updates Jenkinsfile to publish TypeScript coverage reports.
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/9/commits
Issue by cZikos
Monday Jul 23, 2018 at 12:39 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/3
cZikos included the following code: https://github.com/gizafoundation/api-layer/pull/3/commits
Issue by plavjanik
Saturday Jul 28, 2018 at 07:33 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/12
This change provides TLS server certificate for testing on localhost. This certificate is used by the gateway-service and other services have it in the truststore. The certificate is signed by a local certificate authority instead of a 3rd party CA and is meant only for development and testing. The process how to create it and import CA into your browser is documented.
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/12/commits
Issue by cZikos
Tuesday Jul 24, 2018 at 14:55 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/8
This is a dummy pull request
No changes added
Do not merge
cZikos included the following code: https://github.com/gizafoundation/api-layer/pull/8/commits
Issue by vojoup
Monday Jul 30, 2018 at 11:21 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/13
vojoup included the following code: https://github.com/gizafoundation/api-layer/pull/13/commits
Issue by vojoup
Thursday Aug 02, 2018 at 09:05 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/22
This PR should make it easy to use local config for services while using 'Run Dashboard' in IntelliJ
vojoup included the following code: https://github.com/gizafoundation/api-layer/pull/22/commits
Enable dynamic registration of zLUX service and ZSS server in API ML.
Enable zLUX and ZSS dependent services to access them thru API ML.
Issue by supmi01
Monday Sep 03, 2018 at 15:21 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/72
Will probably require server.xml and web ui code changes to update the api format.
Issue by varma09
Monday Oct 01, 2018 at 07:30 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/141
As a Zowe developer, I would like to use the same prefix for packages and configuration properties so I can be consistent with the rest of Zowe and quickly see what code belongs to Zowe.
Acceptance Criteria:
Java and JavaScript packages for APIMIL start with "org.zowe.apiml"
Configuration options in our YAML and properties start with "apiml" instead of "mfaas" - there will be probably a story in HTTPS feature that will do it (this AC is satisfied when this story is defined by Petr)
All references to MfaaS or Mainframe as a Service are removed
Updated enabler are released with a higher number so they are not automatically picked up by existing adopter (e.g. FileMaster)
A utility to quickly change it in the code for existing adopters is created (ideally tested on our code)
Notes:
Issue by supmi01
Wednesday Sep 12, 2018 at 15:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/102
None
Issue by supmi01
Friday Aug 31, 2018 at 08:42 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/48
CA-proprietary UI library is replaced with open Mineral-UI
Issue by varma09
Wednesday Sep 05, 2018 at 13:24 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/88
As Sherman, I would like to define CA REST APIs using static definitions easily without understanding what URLs need to be specified
List of products: based on customer feedback
Issue by daveking999
Wednesday Aug 01, 2018 at 10:18 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/18
The Hystrix timeout was set to the same value as the ribbon timeout.
This is causing unnecessary messages to be sent to the logs...
2018-08-01 12:10:42.815 WARN 21040 --- [.1-10010-exec-9] o.s.c.n.z.f.r.s.AbstractRibbonCommand : The Hystrix timeout of 30000ms for the command caapicatalog is set lower than the combination of the Ribbon read and connect timeout, 120000ms.
The correct value (according to netflix documentation) should be:
ribbon.ConnectTimeout + ribbon.ReadTimeout) * (ribbon.MaxAutoRetries(default = 0) + 1) * (ribbon.MaxAutoRetriesNextServer(default = 1) + 1)
The current ribbon default is 30,000 therefore the hystrix default should be 240,000 millis
Doc: Checking with Andrew if this parameter should be documented
daveking999 included the following code: https://github.com/gizafoundation/api-layer/pull/18/commits
Issue by MarkAckert
Wednesday Jul 18, 2018 at 20:54 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/2
Builds a zip with the proper JARs and scripts needed to install mediation within Zowe. Expect this zip to be pulled in the Zowe PAX build, and converted to a PAX format for now.
MarkAckert included the following code: https://github.com/gizafoundation/api-layer/pull/2/commits
Issue by plavjanik
Wednesday Sep 26, 2018 at 12:55 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/127
As a developer of REST API service, I would like to register all APIs that it provides into API Catalog so the user of the APIs can see individual APIs and use their documentation.
Acceptance criteria:
Issue by vojoup
Tuesday Jul 31, 2018 at 11:57 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/15
Update links in CONTRIBUTING.md
vojoup included the following code: https://github.com/gizafoundation/api-layer/pull/15/commits
Issue by supmi01
Wednesday Sep 12, 2018 at 15:25 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/103
As an application developer, I would like to documentation of public APIs that are available in
Acceptance criteria:
Architectural requirements:
Issue by plavjanik
Tuesday Jul 31, 2018 at 13:18 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/16
Removes old certificates that were signed by CA Technologies
plavjanik included the following code: https://github.com/gizafoundation/api-layer/pull/16/commits
Issue by cZikos
Tuesday Jul 24, 2018 at 12:03 GMT
Originally opened as https://github.com/gizafoundation/api-layer/pull/7
cZikos included the following code: https://github.com/gizafoundation/api-layer/pull/7/commits
Issue by supmi01
Monday Sep 03, 2018 at 15:20 GMT
Originally opened as https://github.com/gizafoundation/api-layer/issues/71
Make the current /Atlas/api endpoints available via a static definition.
As part of this I think we'll need to restructure them to fit the api/v1/explorer path - this will require liberty server config and web ui code changes
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.