jvm-operators / abstract-operator Goto Github PK

Describe the problem:

• When running an operator, I periodically getting
  Mar 24, 2019 1:16:24 AM okhttp3.internal.platform.Platform log
  WARNING: A connection to https://172.30.0.1/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

Any insight?

Steps to reproduce:

Observed Results:

• What happened? This could be a description, log output, etc.

Expected Results:

• What did you expect to happen?

latest version or all possible versions

it could install CRDs there or additional RBAC (if it has rights)

https://github.com/operator-framework/operator-lifecycle-manager/tree/master/deploy/chart/catalog_resources/rh-operators

Describe the problem:

CRD based operator tries to register the CRDs if it's not able to find them in the K8s (here). In the same method, the
io.fabric8.kubernetes.internal.KubernetesDeserializer.registerCustomKind(newPrefix + "/" + crd.getSpec().getVersion() + "#" + this.entityName, InfoClass.class); is called so that fabric8 client knows how to serialize and deserialize the new custom resources. However, if the operator is killed and K8s starts it again, it does not call the line with the registerCustomKind because it finds the existing CRD and returns from the initCrd method (here)

Steps to reproduce:

1. start CRD based operator
2. kill the operator pod
3. wait till operator is again up and running
4. create cluster

This comment is the same issue on the spark-operator side.

So calling the registerCustomKind on some different place should solve the issue. Perhaps line 193?

Describe the problem:

one can specify the env variable OPERATOR_OPERATION_TIMEOUT_MS (defaults to 60 seconds), but this isn't used anywhere in the code. It should be passed to the fabric8 client.

It appears that using Java 12, Reflections library does not detect the @operator'ed class and says there are none. Any ideas? Probably Reflections does not support Java 12?

Best,

Mario

Describe the problem:

The operator creates the CRDs only if it can't find them in the K8s. However, it uses only names of the CRDs to verify if it's there or not. It should also check the group or even version.

Steps to reproduce:

in case of spark-operator which uses this library:

1. https://raw.githubusercontent.com/GoogleCloudPlatform/spark-on-k8s-operator/master/manifest/spark-operator-crds.yaml
2. start the spark operator in CRD mode
3. oc get crd

Observed Results:

NAME                                              AGE
scheduledsparkapplications.sparkoperator.k8s.io   1m
sparkapplications.sparkoperator.k8s.io            1m
sparkclusters.radanalytics.io                     20s  

Expected Results:

something like:

NAME                                              AGE
scheduledsparkapplications.sparkoperator.k8s.io   1m
sparkapplications.sparkoperator.k8s.io            1m
sparkclusters.radanalytics.io                     20s  
sparkapplications.radanalytics.io                 20s  

make the annotation even simpler to provide a default value for the prefix

Describe the problem:

https://github.com/jvm-operators/abstract-operator/blob/master/src/main/java/io/radanalytics/operator/common/CustomResourceWatcher.java#L86

is String namespace = info.getMetadata().getName();
where it should be
String namespace = info.getMetadata().getNamespace();

Describe the problem:

• I have a Flink operator https://github.com/lightbend/fdp-flink-operator build on the same framework. Feel free to add this to the reference implementation and comment on it

Steps to reproduce:

Observed Results:

• What happened? This could be a description, log output, etc.

Expected Results:

• What did you expect to happen?

This is quite important, because it serves as a library.

It appears after having an API call to the kubernetes API, there is a log with the following message:

WARNING: A connection to https://10.96.0.1/ was leaked. Did you forget to close a response body? To see where this was allocated, set the OkHttpClient logger level to FINE: Logger.getLogger(OkHttpClient.class.getName()).setLevel(Level.FINE);

(10.96.0.1) is the kubernetes service in my local minikube.

Perhaps update to the latest kubernetes-client by fabric8?

https://issues.sonatype.org/browse/OSSRH-41606

Dependabot couldn't authenticate with https://oss.sonatype.org/content/repositories/releases.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

You can mention @dependabot in the comments below to contact the Dependabot team.

Describe the problem:

The problem is, I have two

@Operator(forKind = ServiceA.class, prefix = "salaboy.org")

and

@Operator(forKind = ServiceB.class, prefix = "salaboy.org")

From Service A onAdd(ServiceA serviceA) I want get check all the instances of ServiceB.
It feels to me that then I need to register manually a Fabric8 Kube APIs watch on a Custom Resource, which push me to do all the manual handling instead of reusing the ServiceB operator.

Steps to reproduce:

Observed Results:

• What happened? This could be a description, log output, etc.

Expected Results:

It will be great if the AbstractOperator provide a client for fetching those resources. By a Client I mean:

private NonNamespaceOperation<ServiceA, ServiceAList, DoneableServiceA, Resource<ServiceA, DoneableServiceA>> serviceACRDClient;

If this exists.. maybe some pointers to the docs would help

It would be nice to have a way to specify the short name in the @Operator annotation that would be then used as the short name for that CRD. In case of SparkCluster and SparkApplication, it could be spc and spa.

Example:

@Operator(forKind = SparkCluster.class, prefix = "radanalytics.io", shortName = "spc")
public class SparkClusterOperator extends AbstractOperator<SparkCluster> {
...

Currently, the schema of the configmap is captured by the POJOs. Perhaps, it would be more appropriate to use json schema and generate those pojos

may be useful:
https://github.com/joelittlejohn/jsonschema2pojo/

Describe the problem:

• While the Git repo gives sample operators its unclear to a beginner to understand how to use the SDK to create an operator from scratch. Can you provide some example?

Steps to reproduce:

1. Whats the pre-requisit setup needed?
2. How to scaffold the operator contents?
3. How to begin with a simple operator?

Observed Results:

• What happened? This could be a description, log output, etc.

Expected Results:

• What did you expect to happen?

Problem is now, the Entrypoint creates a new DefaultKubernetesClient(), without taking into account if you use CRD's that are protected by RBAC. Is there a way to configure RBAC's certificates in the KubernetesClient with the current implementation?

Best,

Mario

Installing oraclejdk8
$ export JAVA_HOME=~/oraclejdk8
$ export PATH="$JAVA_HOME/bin:$PATH"
$ /bin/install-jdk.sh --target "/home/travis/oraclejdk8" --workspace "/home/travis/.cache/install-jdk" --feature "8" --license "BCL"
Ignoring license option: BCL -- using GPLv2+CE by default
install-jdk.sh 2019-07-17
Expected feature release number in range of 9 to 14, but got: 8
The command "/bin/install-jdk.sh --target "/home/travis/oraclejdk8" --workspace "/home/travis/.cache/install-jdk" --feature "8" --license "BCL"" failed and exited with 3 during .

related issue: https://travis-ci.community/t/error-installing-oraclejdk8-expected-feature-release-number-in-range-of-9-to-14-but-got-8/3766/7

thanks to @geoand this PR quarkusio/quarkus#2910 has been merged and now it's much more convenient to use quarkus w/ the k8s client (and transitively the native image from graalvm)

So let's rebase the SDK on Quarkus, it also allows to use the CDI subset. And potentially we can get rid of the way how we find the classes that have the @Operator annotation and use the programmatic lookup via @Instance

we should be safe here and surround it with try-catch block, because the convert here may throw exception.. For instance if someone puts the same labels the operator is watching for on a wrong config map (with different "shape").

The result of kubectl get X can be enhanced with additionalPrinterColumns specified in the CRD manifest. We should allow the same functionality using the @Operator annotation.

In the latest version, class SDKEntrypoint replacing EntryPoint does not have a main method now.
Do I need to implement my own main class, that invokes SDKEntrypoint?
Is there example of such main somewhere?

Describe the problem:

It seems like this project https://github.com/microbean/microbean-kubernetes-controller/tree/master/src/main/java/org/microbean/kubernetes/controller provides very similar functionality. Did you try to compare the two implementations? One difference is that micro bean implements a queue between watchers and execution. Is this something worth exploring?

@Operator is easy to use in Java, Scala and Kotlin, but for instance in JavaScript (Nashorn) there is currently no support

1. create the uber jar w/ abstract operator

rlwrap jrunscript -cp ./abstract-operator-0.2.2-SNAPSHOT.jar

var AbstractOperator = Java.type("io.radanalytics.operator.common.AbstractOperator");
var MyOperator = Java.extend(AbstractOperator, { onAdd: function(foo) { print("added " + foo); }, onDelete: function(foo) { print("deleted " + foo); } });
new MyOperator()

java.lang.NullPointerException
	at io.radanalytics.operator.common.AbstractOperator.<init>(AbstractOperator.java:51)

(annotation)

Describe the problem:

• CRD contains namespace for every resource that it creates (I can see it using describe). I added a namespace to the EntityInfo assuming that during reconciliation it would be read from the CRD, but I am reading null. Am I missing something?

Steps to reproduce:

1. Add namespace to EntityInfo class
2. Create resources
3. Log desired resources during reconciliation

Observed Results:

• namespace is null.

Expected Results:

• Namespace is the namespace that can be obtained from oc describe resource

Describe the problem:

• On OpenShift 4, the framework does not recognize OpenShift:
  2019-04-18 19:33:39 INFO Entrypoint:217 - https://172.30.0.1/oapi returned 404. We are not on OpenShift. Assuming, we are on Kubernetes. As a result, Kubernetes not OS client is used and everything fails

Steps to reproduce:

1. Try to run operator on OpenShift 4

Observed Results:

VM settings:
Max. Heap Size (Estimated): 3.76G
Ergonomics Machine Class: server
Using VM: OpenJDK 64-Bit Server VM

2019-04-18 19:33:38 INFO Entrypoint:56 - Starting..
2019-04-18 19:33:39 INFO Entrypoint:217 - https://172.30.0.1/oapi returned 404. We are not on OpenShift. Assuming, we are on Kubernetes.
2019-04-18 19:33:39 INFO Manifests:193 - 55 attributes loaded from 52 stream(s) in 51ms, 55 saved, 724 ignored: ["Ant-Version", "Archiver-Version", "Automatic-Module-Name", "Bnd-LastModified", "Build-Jdk", "Build-Timestamp", "Built-By", "Bundle-ClassPath", "Bundle-ContactAddress", "Bundle-Description", "Bundle-DocURL", "Bundle-License", "Bundle-ManifestVersion", "Bundle-Name", "Bundle-RequiredExecutionEnvironment", "Bundle-SymbolicName", "Bundle-Vendor", "Bundle-Version", "Created-By", "DynamicImport-Package", "Export-Package", "Extension-Name", "Fragment-Host", "Ignore-Package", "Implementation-Build", "Implementation-Build-Date", "Implementation-Title", "Implementation-URL", "Implementation-Vendor", "Implementation-Vendor-Id", "Implementation-Version", "Import-Package", "Include-Resource", "JCabi-Build", "JCabi-Date", "JCabi-Version", "Java-Vendor", "Java-Version", "Main-Class", "Manifest-Version", "Os-Arch", "Os-Name", "Os-Version", "Package", "Private-Package", "Require-Capability", "Scm-Connection", "Scm-Revision", "Scm-Url", "Specification-Title", "Specification-Vendor", "Specification-Version", "Tool", "X-Compile-Source-JDK", "X-Compile-Target-JDK"]
2019-04-18 19:33:39 INFO Entrypoint:236 -
�[31mOperator�[0m has started in version �[32m0.6.1�[0m.

2019-04-18 19:33:39 INFO Entrypoint:239 - Git sha: �[33mtags/LANG_3_2_1_RC1@r1555582; 2014-01-05 18:01:10+0100�[0m
2019-04-18 19:33:39 INFO Entrypoint:241 - ==================

2019-04-18 19:33:39 INFO Entrypoint:76 - �[33mKubernetes�[0m environment detected.
2019-04-18 19:33:43 INFO AbstractOperator:246 - Starting 'FlinkCluster' operator for namespace flink1
Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:64)
at io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:53)
at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:238)
at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:191)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:384)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:328)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.listRequestHelper(BaseOperation.java:165)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:607)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.list(BaseOperation.java:67)
at io.radanalytics.operator.common.crd.CrdDeployer.initCrds(CrdDeployer.java:34)
at io.radanalytics.operator.common.AbstractOperator.start(AbstractOperator.java:249)
at io.radanalytics.operator.Entrypoint.lambda$runForNamespace$6(Entrypoint.java:153)
at java.util.stream.Streams$RangeIntSpliterator.forEachRemaining(Streams.java:110)
at java.util.stream.IntPipeline$Head.forEach(IntPipeline.java:557)
at io.radanalytics.operator.Entrypoint.runForNamespace(Entrypoint.java:136)
at io.radanalytics.operator.Entrypoint.run(Entrypoint.java:92)
at io.radanalytics.operator.Entrypoint.main(Entrypoint.java:60)
Caused by: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "nullable" (class io.fabric8.kubernetes.api.model.apiextensions.JSONSchemaProps), not marked as ignorable (36 known properties: "definitions", "default", "$schema", "multipleOf", "minItems", "exclusiveMinimum", "not", "allOf", "additionalItems", "id", "oneOf", "required", "maximum", "exclusiveMaximum", "anyOf", "minProperties", "externalDocs", "dependencies", "example", "$ref", "properties", "format", "additionalProperties", "description", "enum", "minimum", "pattern", "minLength", "patternProperties", "maxLength", "uniqueItems", "maxProperties", "type", "maxItems", "items", "title"])
at [Source: (BufferedInputStream); line: 1, column: 202859] (through reference chain: io.fabric8.kubernetes.api.model.apiextensions.CustomResourceDefinitionList["items"]->java.util.ArrayList[31]->io.fabric8.kubernetes.api.model.apiextensions.CustomResourceDefinition["spec"]->io.fabric8.kubernetes.api.model.apiextensions.CustomResourceDefinitionSpec["validation"]->io.fabric8.kubernetes.api.model.apiextensions.CustomResourceValidation["openAPIV3Schema"]->io.fabric8.kubernetes.api.model.apiextensions.JSONSchemaProps["properties"]->java.util.LinkedHashMap["spec"]->io.fabric8.kubernetes.api.model.apiextensions.JSONSchemaProps["properties"]->java.util.LinkedHashMap["observedConfig"]->io.fabric8.kubernetes.api.model.apiextensions.JSONSchemaProps["nullable"])
at com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException.from(UnrecognizedPropertyException.java:61)
at com.fasterxml.jackson.databind.DeserializationContext.handleUnknownProperty(DeserializationContext.java:823)
at com.fasterxml.jackson.databind.deser.std.StdDeserializer.handleUnknownProperty(StdDeserializer.java:1153)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownProperty(BeanDeserializerBase.java:1589)
at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.handleUnknownVanilla(BeanDeserializerBase.java:1567)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:294)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:527)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:364)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:29)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:286)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:245)
at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:27)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:127)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:288)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:151)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4013)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3077)
at io.fabric8.kubernetes.client.utils.Serialization.unmarshal(Serialization.java:236)
... 15 more

Expected Results:

• Normal execution

it may be contraproductive to have these special symbols in the log file

https://travis-ci.org/jvm-operators/abstract-operator/jobs/427275779#L2998