pravega / pravega-keycloak Goto Github PK
View Code? Open in Web Editor NEWKeycloak Security Plugin for Pravega
License: Apache License 2.0
Keycloak Security Plugin for Pravega
License: Apache License 2.0
Problem
A previous PR had brought some robustness changes whereby if the requests to Keycloak responded with certain exceptions, the request would be retried, while others would result in a fatal/terminal state.
A particular exception was not added to the list of explicitly retried situations: "java.net.UnknownHostException".
This should ideally be retried since there are situations where network problems (DNS) occur and name resolution temporarily fails.
Solution
Catch java.net.UnknownHostException here
Problem description
Travis CI builds are unable to publish snapshots to JFrog, due to bad credentials.
The code referenced in the Problem Location
throws this error:
Could not PUT 'https://oss.jfrog.org/oss-snapshot-local/io/pravega/pravega-keycloak-client/0.8.0-16.ef203a4-SNAPSHOT/maven-metadata.xml'. Received status code 401 from server:
Problem location
Line 33 in ef203a4
Suggestions for an improvement
Update credentials to a new set.
Problem description
As part of 0.8.0 release, we need to update the build version in master.
Problem location
Gradle properties.
Suggestions for an improvement
Update the Pravega version to 0.9.0-SNAPSHOT.
There is no way to pass the keycloak JSON file content as a string in the client configuration.
Problem description
Update Pravega dependency version in branch r0.8 and update the release version.
Problem location
gradle.properties
Suggestions for an improvement
Develop a plugin for Pravega client to authenticate with Keycloak.
In KeycloakAuthzClient, the default try policy for http error is max 20 times, for 1st attempt wait 100ms, for following attempts double wait time of previous one.
With this setting at 20th attempt client will wait 2^19 *100 ms = 14.56h before retrying.
We've got a system once had a DNS issue to keycloak server. When the DNS issue was resolved, the client was stuck at19th retry. And client could not recover without waiting another 7 hours.
It is true that KeycloakAuthzClientBuild has offered method to override default http retry configs, but the option is not viable to be configured via EventStreamClientFactory via PravegaKeycloakCredentials.
There should be a reasonable default upper limit wait time set in KeycloakAuthzClient itself to limit it to minutes maximum.
Update Keycloak versioned libraries to Keycloak 15.0.2 Restest unit tests and test against Keycloak.
Bump up the version on master to 0.14.0-SNAPSHOT in gradle.properties to reflect dependancy on Pravega 0.14.0
Problem description:
Need an update for keycloak client dependency to version 21.1.2 to support Keycloak 21.
Update location:
Gradle.properties file and new API in Keycloak 21.
Update the buildVersion to 0.9.1-SNAPSHOT in the r0.9 to setup potential future builds with bug fixes.
Problem description:
Need an update for Keycloak client dependency to version 23.0.4 to support Keycloak 23.
Update location:
gradle.properties
gradle/wrapper/gradle-wrapper.properties
build.gradle
Problem description
Update Pravega dependency version in branch r0.9 and update the release version.
Problem location
gradle.properties
Suggestions for an improvement
Update pravegaVersion to 0.9.0 and buildVersion to 0.9.0
Problem description
Travis CI builds are unable to publish snapshots to JFrog, due to bad credentials.
The code referenced in the Problem Location
throws this error:
* What went wrong:
Execution failed for task ':client:publishMavenJavaPublicationToJcenterSnapshotRepository'.
> Failed to publish publication 'mavenJava' to repository 'jcenterSnapshot'
> Could not PUT 'https://oss.jfrog.org/oss-snapshot-local/io/pravega/pravega-keycloak-client/0.8.0-32.ac17464-SNAPSHOT/maven-metadata.xml'. Received status code 401 from server:
Problem location
Lines 29 to 30 in ac17464
Suggestions for an improvement
Update credentials to a new set.
Task: The PravegaKeycloakCredentials library uses Keycloak 10.0.2 AuthzClient.
As part of keycloak-12 upgrade needs to be updated using 12.0.4 libraries and tested.
Testing:
After the Jar is created with the updated libraries test with Keycloak 12 server and verify if client receives tokens.
Problem description
A typical name of the artifact should be something like 0.10.0-1.ff9f6d1-SNAPSHOT
. With the [version]-[commit count]-[commit sha]-SNAPSHOT
format. But for artifacts produced by Github Actions, the commit count are always set to 1 now. This is caused by the depth==1
clone by default.
ref: First bullet point in https://github.com/marketplace/actions/checkout#whats-new
Problem location
./.github/workflows/build.yml
Suggestions for an improvement
Fetch all the commits not only the HEAD
from the repo.
you can refer to the Flink connector PR: pravega/flink-connectors#492
This repo repository has not been doing releases in a way that is aligned with other Pravega repos like Pravega and Pravega Flink Connector. It uses a release process which is very different from how rest of the Pravega repos do. Among other things:
Problem description
As part of 0.12.0 release, we need to update the build version in master.
Problem location
Gradle properties.
Suggestions for an improvement
Update the Pravega version and set Pravega Keycloak version to 0.13.0-SNAPSHOT.
Since a new release is available, pravegaVersion
and buildVersion
need to be updated in master.
Reference: https://github.com/pravega/pravega-keycloak/wiki/How-to-Release.
pravega-keycloak needs to be released against version 0.12.0 of Pravega.
This line is introduced in this commit: 6f44fb5, and this guava dependency is from the pravega-shared-security
which is a compileOnly
dependency in keycloak client. This means application needs to have guava
in the classpath, but unfortunately Flink connector has shaded guava
to avoid conflict
https://github.com/pravega/flink-connectors/blob/53666981e3d42a4de7a28686f562a18d69410620/build.gradle#L156
This makes me have java.lang.ClassNotFoundException: com.google.common.base.Strings
when I try to upgrade the keycloak client and run the Flink jobs. Although this could be fixed by the application to explicitly add guava
, but I think it's better to fix in this project.
Problem description
As part of 0.11.0 release, we need to update the build version in master.
Problem location
Gradle properties.
Suggestions for an improvement
Update the Pravega version and set Pravega Keycloak version to 0.12.0-SNAPSHOT.
Problem description
Update Pravega dependency version in branch r0.11 and update the release version.
Problem location
gradle.properties
Suggestions for an improvement
Overview
Network problems can occur anytime and manifest themselves through a variety of not always predictable exceptions types, and it's important to have resilience built-in when trying to reach Keycloak to obtain tokens.
There is such resilience built-in in the KeycloakAuthzClient.java: https://github.com/pravega/pravega-keycloak/blob/master/client/src/main/java/io/pravega/keycloak/client/KeycloakAuthzClient.java#L127 where we carefully decide which family of exceptions should be retried and which shouldn't.
The problem is we try too hard to guess which exact ones should be retryable and we which shouldn't, and we also take an approach of "if I don't know this exception, don't retry it". This means that from time to time, strange exceptions occur if in an SSL environment, which get wrapped into RuntimeExceptions, sometimes not etc. It makes it very hard to predict what can be retried and what can't.
One example of a new one that was observed recently:
ERROR [2021-08-26 20:49:22.164] [grpc-default-executor-138] i.p.k.client.KeycloakAuthzClient: Other non retryable exception
java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:167)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:109)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1392)
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1300)
As well as: javax.net.ssl.SSLHandshakeException: Remote host terminated the handshake
Both of these should ideally be retried.
(note that other exceptions can also occur in non SSL environments)
Solution
Only use a very small set of exceptions that you know for sure should not be retried such as authentication errors. For anything else, let it be. It'll get retried for the max amount of retries configured. Most likely it was meant to be retried. If not, it'll just eventually fail.
Problem location
Solution
Zero out the few known exceptions we should not retry, and let everything else be retried by default.
Since a new release v0.7.0
is available, pravegaVersion and buildVersion need to be updated in master. (Reference: https://github.com/pravega/pravega-keycloak/wiki/How-to-Release.)
Problem description
The Keycloak client that is used to obtain access tokens and relying party tokens (RPTs) can run into connection problems making these REST requests to Keycloak. When this happens, the Pravega client which uses this implementation of the Credentials interface does not get a chance to retry the attempts to retrieve a token since it doesn't know what the cause (straight authentication errors should not be retried while other http errors should)
Expected behavior
If Keycloak is unreachable when a token request is sent, the request should be retried a certain number of times. After the number of retries is exceeded can the exception actually be bubbled up to the Credentials.getAuthenticationToken() call.
Problem description:
Need an update for keycloak client dependency to version 19.0.3 to support Keycloak 19.
Update location:
Gradle.properties file
As part of pravega/pravega#5339, the Credentials
interface location changed. We need to update the dependencies in this repo to pick up the latest Pravega version, adjust the package location for Credentials
and remove dependency on the pravega client itself which is no longer necessary.
Task: The PravegaKeycloakCredentials library uses Keycloak 6.0.1 AuthzClient.
As part of keycloak-10 upgrade needs to be updated using 10.0.2 libraries and retested.
Testing:
This can be tested updating the libraries pravega-keycloak libraries, once Issue#12 is fixed
After the Jar is created with the updated libraries test with Keycloak 10 server and verify if client receives tokens.
Problem description
Update Pravega dependency version in branch r0.10 to take in pravega interfaces from Pravega 0.10.1
Update this build Version to 0.10.1
Problem location
gradle.properties
Overview
We need to migrate to using GitHub actions for regular builds to be inline with the rest of Pravega components.
Additionally, we need to publish snapshots to GitHub packages instead of JCenter.
Regular releases will still be published to Maven Central as usual.
Where
Need to add a github workflow and remove travis ci yaml file.
Problem description:
Need an update for keycloak client dependency to version 18.0.2 to support Keycloak 18.
Update location:
Gradle.properties file
Problem description
As part of 0.9.0 release, we need to update the build version in master.
Problem location
Gradle properties.
Suggestions for an improvement
Update the buildVersion to 0.10.0-SNAPSHOT.
Update the pravegaVersion to 0.9.0
Problem description
Update Pravega dependency version in master branch to take in pravega interfaces from Pravega 0.10.1
Update this build Version to 0.11.0-SNAPSHOT
Problem location
gradle.properties
To be able to publish to jfrog snapshot repository (OJO), we need to update the bintray credentials.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.