Description of the feature or enhancement:

This is a request to return the generated filenames from the synth() function.

Use Case:

Our usecase involves managing the full lifecycle of the resource after synth programmatically, and at the moment we're needing to deduce the output filename for a given App by looking in the outdir directory and asserting that there is only a single file for each Chart. Alternately, we could reproduce the logic from the link above in code, but that runs the risk of a changing naming scheme in cdk8s breaking our code at some point.

Proposed Solution:

  /**
   * Synthesizes all manifests to the output directory
   */
  public synth(): { [key: string]: string } {

    fs.mkdirSync(this.outdir, { recursive: true });

    // this is kind of sucky, eventually I would like the DependencyGraph
    // to be able to answer this question.
    const hasDependantCharts = resolveDependencies(this);

    // Since we plan on removing the distributed synth mechanism, we no longer call `Node.synthesize`, but rather simply implement
    // the necessary operations. We do however want to preserve the distributed validation.
    validate(this);

    const simpleManifestNamer = (chart: Chart) => `${Node.of(chart).uniqueId}.k8s.yaml`;
    const manifestNamer = hasDependantCharts ? (chart: Chart) => `${index.toString().padStart(4, '0')}-${simpleManifestNamer(chart)}` : simpleManifestNamer;

    const charts: IConstruct[] = new DependencyGraph(Node.of(this)).topology().filter(x => x instanceof Chart);

    let index = 0;
    const outputNames: { [key: string]: string } = [];
    for (const node of charts) {
      const chart: Chart = Chart.of(node);
      const outputFileName: string = path.join(this.outdir, manifestNamer(chart));
      outputNames[chart.toString()] = outputFileName
      Yaml.save(outputFileName, chartToKube(chart));
      index++;
    }
    return outputNames;

  }

Other:

We are using the python interface, so we would need this return value to be available via the wrapped python function as well. I think that should work seamlessly, but would need to investigate the jsii interaction a bit before feeling confident about that.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

Description of the bug:

Hello

Usage of ApiObjectMetadataDefinition in object definition generated unexpected key output on synth

According to my tests, its only occur with CRDs

Reproduction Steps:

• Create a new project in Python
• Import CRD with cdk8s import argocd-crd.yaml
• Use the CRD in a Chart
• cdk8s synth

#!/usr/bin/env python
from constructs import Construct
from cdk8s import App, Chart, ApiObjectMetadataDefinition
from imports import k8s
from imports.io import argoproj


from imports import k8s


class MyChart(Chart):
    def __init__(self, scope: Construct, id: str):
        super().__init__(scope, id)

        argo_namespace="aa"
        labels={
            "app": "argocd"
        }
        k8s_server="https://kubernetes.default.svc"
        namespace="default"
        name="app"
        repo_url="http://xxx.gihub.com"
        path="."
        target_revision="master"


        k8s.KubePod(self, "pod", 
            metadata=ApiObjectMetadataDefinition(
                name=name,
                namespace=namespace,
            ),
            spec=k8s.PodSpec(
                containers=[
                    k8s.Container(
                        name="test",
                        image="nginx:latest"
                    )
                ]
            )
        )

        # define resources here
        argoproj.Application(
            self,
            "argocdapp",
             metadata=ApiObjectMetadataDefinition(
                name=name,
                namespace=namespace,
            ),
            spec=argoproj.ApplicationSpec(
                destination=argoproj.ApplicationSpecDestination(
                    namespace=namespace, server=k8s_server
                ),
                
                sync_policy=argoproj.ApplicationSpecSyncPolicy(
                    automated=argoproj.ApplicationSpecSyncPolicyAutomated(
                    self_heal=True,
                    )
                ),
                project=name,
                source=argoproj.ApplicationSpecSource(
                    repo_url=repo_url,
                    path=path,
                    target_revision=target_revision,
                )               
            ),
        )


app = App()
MyChart(app, "cdk8s-import-crd-python")

app.synth()

Error Log:

Output:

apiVersion: v1
kind: Pod
metadata:
  name: app
  namespace: default
spec:
  containers:
    - image: nginx:latest
      name: test
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  _additionalAttributes:
    name: app
    namespace: default
  name: app
  namespace: default
spec:
  destination:
    namespace: default
    server: https://kubernetes.default.svc
  project: app
  source:
    path: "."
    repoURL: http://xxx.gihub.com
    targetRevision: master
  syncPolicy:
    automated:
      selfHeal: true

The portion not expected is

  _additionalAttributes:
    name: app
    namespace: default

Environment:

• Framework Version: 1.0.21
• OS: Linux

Other:

Workaroud

Define the metadata definition with:

metadata={"name": name,"namespace": namespace},

Thanks

This is 🐛 Bug Report

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625297845

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625282069

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625297845

Description of the bug:

When nesting charts within a parent chart, applying the parent chart to an app and then synthesizing the app, I'm seeing duplicated synthed resources - one file for each separate chart nested in the parent chart, and then one file for the parent chart which also includes the resources from the nested charts.

This is problematic when applying to Kubernetes using kubectl apply -f dist/ as the files contain duplicate identically named resources. Not sure if this is expected or not but submitting as a bug report just in case 🙂

Reproduction Steps:

app/test.ts:

import { App, Chart } from 'cdk8s'
import { Construct } from 'constructs'
import { Namespace } from '../imports/k8s';


export class ChildChart1 extends Chart {
    constructor(scope: Construct, name: string) {
      super(scope, name)
      const ns1 = new Namespace(this, 'namespace1')
    }
}

export class ChildChart2 extends Chart {
    constructor(scope: Construct, name: string) {
      super(scope, name)
      const ns2 = new Namespace(this, 'namespace2')
    }
}

export class ParentChart extends Chart {
    constructor(scope: Construct, name: string) {
        super(scope, name)
        const childChart1 = new ChildChart1(this, 'child1')
        const childChart2 = new ChildChart2(this, 'child2')
        const ns3 = new Namespace(this, 'namespace3')
    }
}
const app = new App()
const parent = new ParentChart(app, 'parent')
app.synth()

shell:

➜  cdk8s git:(feat/nested-children) ✗ cdk8s synth -a "npx ts-node -r tsconfig-paths/register app/test.ts"
dist/parent-child1-cad88a69.k8s.yaml
dist/parent-child2-1e516eb8.k8s.yaml
dist/parent.k8s.yaml

➜  cdk8s git:(feat/nested-children) ✗ cat dist/parent-child1-cad88a69.k8s.yaml
apiVersion: "v1"
kind: "Namespace"
metadata:
  name: "parent-child1-namespace1-e27612e5"

➜  cdk8s git:(feat/nested-children) ✗ cat dist/parent-child2-1e516eb8.k8s.yaml
apiVersion: "v1"
kind: "Namespace"
metadata:
  name: "parent-child2-namespace2-32aecabe"

➜  cdk8s git:(feat/nested-children) ✗ cat dist/parent.k8s.yaml
apiVersion: "v1"
kind: "Namespace"
metadata:
  name: "parent-child1-namespace1-e27612e5"
---
apiVersion: "v1"
kind: "Namespace"
metadata:
  name: "parent-child2-namespace2-32aecabe"
---
apiVersion: "v1"
kind: "Namespace"
metadata:
  name: "parent-namespace3-7362c69c"

Environment:

[email protected] 
├── @types/[email protected]
├── @types/[email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
├── [email protected]
└── [email protected]

Other:

I'm not bothered what the correct output is here. It seems to me that output should either be in individual files (one per nested child, with the parent resources in their own file) or all resources should be put into the one parent file. I don't think it matters, as long as the directory itself doesn't contain duplicate resources.

This is 🐛 Bug Report

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1594600700

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628699315

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1544238192

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625282069

Parent issue: https://github.com/cdk8s-team/cdk8s/issues/571
Original pull request: cdk8s-team/cdk8s#572

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625297845

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628716140

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625282069

It is sometimes desirable to be able to reference values from other resources during synthesis.

From the README of IBM's Composable Operator:

Kubernetes object specifications often require constant values for their fields. When deploying an entire application with many different resources, this limitation often results in the need for staged deployments, because some resources have to be deployed first in order to determine what data to provide for the specifications of dependent resources. This undermines the declarative nature of Kubernetes object specification and requires workflows, manual step-by-step instructions and/or brittle automated scripts for the deployment of applications as a whole.

We should consider two types of references:

• Runtime: Reference values from a different deployment. This means that we don't know the value during synthesis and we need to wait for it to become available.
• Synth-time: Reference values within your CDK app that can be resolved during synthesis. This means that the data is available during synthesis and we just want to be able to reference a configuration value of another resources.

Runtime resolution requires something like wrapping the resources using the Composable Operator which will wait until the data is available before it creates the resource.

Synth-time resolution can be achieved today by passing the actual value itself to the consumption site, or we can offer a read API for generated resources (i.e. myService.spec.ports[0].port).

Questions:

• For synth-time resolution, does it makes sense to automatically add a dependency between the resources (see cdk8s-team/cdk8s#111) when the reference it taken.
• For runtime resolution, can we offer an API that automatically synthesizes runtime resolution if the value doesn't exist (myService.state.foo.bar). What's the idiomatic way to "import" resources?

Can we offer something like this (based on one of their examples)?

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628699315

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1556568136

Since the release of cdk8s for golang, I've been using cdk8s inside server-side applications.

The app.synth() function is of type void so it returns no value, it only creates files depending on the YAML output type.

It would be great to be able to have app.synth() return a string value of our app in YAML, like this we can avoid reading the file after app.synth() and directly grab the value of the YAML like myAppYaml = app.synth().

This feature would help integrate cdk8s inside server-side applications very nicely.

slack message: https://cdk-dev.slack.com/archives/C0184GCBY4X/p1638274810058800

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628699315

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625282069

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1589956498

Hello, is there any changelog available or planned? It would be nice to know what new releases contain before upgrading to them thanks!

It would be cool to have the option to synth the manifests per resource instead of per chart.
It could be a dump of all manifests, or even create folders per chart and place the manifests in there.

I would propose that we add outputYamlFormat option in the cdk8s.yaml config file that accepts these values:

• filePerChart: Current behavior - defaults to this if not specified otherwise
• folderPerChart: Creates a folder with the Chart name and writes the Resources such as deployment.yaml, services.yaml, and so on.
• filePerResource: dump of a file for each resource on the dist folder

For the folderPerChart option we could assume the chart name as the folder name and for the filePerResource, we can assume some convention, like chartName-resourcekind, e.g. myapp-deployment, my-app-service and so on.
If there are multiple resources of the same kind, they can stay within the same yaml file as now to avoid file naming problems.

Thoughts?

At the moment, these types are immutable. All their properties have to be defined upon initialization. Similarly to the AWS CDK's "cfn resources", we should allow users to mutate these objects to improve authoring experience.

Let's say I want to conditionally determine the value of a couple of fields in an imported k8s object. Today, I would need to do it like this:

new MyKind(this, 'foo', {
  spec: {
    field1: myCondition ? 'valueA' : 'valueB',
    sub: {
      tree: myCondition ? 'valueA' : 'valueB'
    }
  }
});

new YourKind(this, 'your', {
  spec: {
    opt: myCondition ? 10  : 20
  }
});

This feature will allow me to write this code like this:

const my = new MyKind(this, 'my');
const your = new YourKind(this, 'your');

if (myCondition) {
  my.spec.field1 = my.sub.tree = 'valueA';
  your.opt = 10;
} else {
  my.spec.field1 = my.sub.tree = 'valueB';
  your.opt = 20;
}

An issue here is who is responsible to initialize sub-objects such as spec and sub.tree. I'll leave it for the implementer to ponder options.

For imported k8s resources, it is possible to specify all supported metadata fields:

from imports import k8s

k8s.KubePod(self, "mypod", metadata={"finalizers": ["my-finalizer"]})

But if you try doing this with an ApiObject:

import cdk8s

cdk8s.ApiObject(
    self,
    "mypod",
    api_version="v1",
    kind="Pod",
    metadata={"finalizers": ["my-finalizer"]}
)

then it produces the error:

/cdk8s/__init__.py", line 375, in __init__
    metadata = ApiObjectMetadata(**metadata)
TypeError: __init__() got an unexpected keyword argument 'finalizers'

The root cause is that ApiObject expects its parameters to match the ApiObjectMetadata interface (link). In TypeScript, ApiObjectMetadata has a field typed as [key: string]: any that allows any value to be inserted (so that we don't have to document every available field), but this does not get translated correctly to Python due to aws/jsii#2928.

Imported objects' metadata have to satisfy the ObjectMeta interface instead of ApiObjectMetadata, and ObjectMeta includes all available metadata fields, so that is why the first example does not error.

Related: cdk8s-team/cdk8s-cli#33

I think the most appropriate short-term fix is to add the missing metadata fields to the ApiObjectMetadata interface.

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1513895613

I asked a question on SO about IDs being appended to file names after running cdk8s synth, but after more digging I discovered some behavior that isn't clear to me and I wasn't able to find anything in the docs.

Here's what I'm seeing:

If I create a new project with cdk8s init typescript-app, then copy-paste one of the Getting-Started examples into main.ts, then run npm run compile && npm run synth, it will generate a file called dist/hello.k8s.yaml. Wonderful!

import { Construct } from 'constructs';
import { App, Chart, ChartProps } from 'cdk8s';

// imported constructs
import { KubeDeployment, KubeService, IntOrString } from './imports/k8s';

export class MyChart extends Chart {
  constructor(scope: Construct, id: string, props: ChartProps = { }) {
    super(scope, id, props);

    const label = { app: 'hello-k8s' };

    new KubeService(this, 'service', {
      spec: {
        type: 'LoadBalancer',
        ports: [ { port: 80, targetPort: IntOrString.fromNumber(8080) } ],
        selector: label
      }
    });

    new KubeDeployment(this, 'deployment', {
      spec: {
        replicas: 2,
        selector: {
          matchLabels: label
        },
        template: {
          metadata: { labels: label },
          spec: {
            containers: [
              {
                name: 'hello-kubernetes',
                image: 'paulbouwer/hello-kubernetes:1.7',
                ports: [ { containerPort: 8080 } ]
              }
            ]
          }
        }
      }
    });
  }
}

const app = new App();
new MyChart(app, 'hello');
app.synth();

The weird behavior happens when I add a period (.) to the ID of the chart. For example, if I change new MyChart(app, 'hello'); to new MyChart(app, 'hello.us-east-1'); and run npm run compile && npm run synth, the generated file is now dist/hello.us-east-1-c835ef5a.k8s.yaml. I would expect the file name to be dist/hello.us-east-1.k8s.yaml.

Is this a bug or a feature? (My gut instinct tells me this is a bug).

If it is a bug, I'll happily reopen this ticket as a bug. If it's a feature, I'd sure appreciate some clarification and insight!

In addition to load() and save() which work with files, we should also expose in-memory versions.

Description of the feature or enhancement:

I would love to have some YAML utilities built-in to cdk8s.

Use Case:

While working with k8s, we are working with YAML in one way or another. Having some YAML utilities would enable users to do things like this one, without having to install external deps:

    const datasources = new ConfigMap(this, 'grafana-datasources', {
      data: {
        "loki.yaml": yaml.safeDump({ 
          apiVersion: 1, 
          datasources: [{
            name: 'Loki', 
            type: 'loki', 
            access: 'proxy', 
            url: `${loki.releaseName}:3100`, 
            jsonData: { maxLines: 1000 } 
          }]
        })
      }
    });

Proposed Solution:

Other:

This was proposed by @eladb on Slack.

• 👋 I may be able to implement this feature request -> I don't have enough bandwidth right now :(
• ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

Description of the feature or enhancement:

The spec property of L1 constructs should be required.

For example, right now one can instantiate a pod like this:

new k8s.KubePod(this, 'Pod');

This doesn't make sense because a spec must be configured, and L1 constructs do not offer post instantiation mutations.

Can we make spec required?

Use Case:

Stronger type validations.

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

Is your feature request related to a problem? Please describe.
I would like to be able to control how the yaml is being outputted

Describe the solution you'd like
Have access to the YAML package options https://eemeli.org/yaml/#options before calling .synth() method.

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625282069

Description of the bug:

This looks like an import bug or maybe I am not using the API correctly. The generated istio CRD for example misses the metadata property:

export interface VirtualServiceProps {
  /**
   * Configuration affecting label/content routing, sni routing, etc. See more details at: https://istio.io/docs/reference/config/networking/virtual-service.html
   *
   * @schema VirtualService#spec
   */
  readonly spec?: VirtualServiceSpec;

}

As a result if have to use @ts-ignore every time I want to provide a name:

const vsvc = new VirtualService(this, 'vsvc', {
    // @ts-ignore
    metadata: {
      name: 'cool-service',
    },
    spec: {...}
});

Ideally, the VirtualService should look like this:

import {ObjectMeta} from './k8s';

export interface VirtualServiceProps {

  /**
   * Standard object metadata.
   *
   * @schema io.k8s.api.apps.v1.Deployment#metadata
   */
  readonly metadata?: ObjectMeta;

  /**
   * Configuration affecting label/content routing, sni routing, etc. See more details at: https://istio.io/docs/reference/config/networking/virtual-service.html
   *
   * @schema VirtualService#spec
   */
  readonly spec?: VirtualServiceSpec;

}

Granted, the metadata property is not present in the schema but all persisted resources have it.

Reproduction Steps:

Run cdk8s imports with any CRD.

Error Log:

Environment:

• Framework Version: 1.0.0-beta.10
• OS: Linux

Other:

This is 🐛 Bug Report

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1423784628

Currently, the JsonPatch class can be used to apply overrides / updates to resources that have already been generated. However, sometimes these modifications do not appear in the generated YAML because of type mismatches, especially when newer kubernetes versions expect IntOrString where they previously only required an int or string.

For example, applying this patch to a KubeDeployment will produce rollingUpdate: {}:

apiObject.addJsonPatch(
    JsonPatch.add("/spec/strategy", { rollingUpdate: { maxSurge: 1, maxUnavailable: 0 }})
);

...while applying this patch to a KubeDeployment will produce rollingUpdate: { maxSurge: 1, maxUnavailable: 0 } (in YAML):

apiObject.addJsonPatch(
    JsonPatch.add("/spec/strategy", {
		rollingUpdate: { maxSurge: IntOrString.fromNumber(1), maxUnavailable: IntOrString.fromNumber(0)}
	})
);

This type of no-op did not occur in older versions of cdk8s, but it does now since cdk8s-team/cdk8s-cli#55, as the toJson() method on the generated KubeDeployment class calls toJson_KubeDeployment(), which discards all values that were not expected (including in this case, values of the wrong type). (The patches get applied in the superclass, ApiObject, before toJson_KubeDeployment() is called).

I think this is problematic since JsonPatch should be able to add any extra content (including from newer / older kubernetes specs) to the ApiObject, not just those expected by the interface, so that users can patch their constructs for older or newer kubernetes versions. It also has a clunky experience (Why did the JSON Patch values not appear? There's no error message!)

Possible solutions:

• modify how json2jsii generates k8s.ts to try and "coerce" unexpected types somehow
• modify how json2jsii generates k8s.ts to do some kind of runtime checking to assert that values are the expected types, and print a warning when unexpected keys or values appear
• change when json patches get applied somehow, so the toJson() method in KubeDeployment instead calls toJsonUnpatched(), then toJson_KubeDeployment(), and then JsonPatch.apply(json, ...this.patches)

I've got an existing yaml resource file I'd like to include:

new Include(this, 'Deployment', {
  url: `../../deployment-${props.env}.yaml`,
});

However, I get an error about not being able to determine the protocol:

Error: unable to determine protocol from url: ../../deployment-staging.yaml

during the _loadurl.js script.

The docs indicate I can point it at a file but I'm not sure how this would every work as the url.parse() will take absolute and relative file pathing and report no protocol

Node v14

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628699315

We would like to offer utilities for unit testing in the cdk8s.Testing class.

Here are some examples of useful assertions:

1. Assert that a chart synthesizes a manifest with a specific API object of a specific kind and optionally: apiVersion, a subset of properties (or exact match)
2. assert count of API objects from a certain kind
3. Assert that a chart synthesizes a manifest that matches a snapshot stored in a file (would be nice to make it easy to update the snapshot by e.g. setting an env. variable).

These functions should be testing-framework agnostic (and naturally jsii-able) so they can be used in all languages. If people want to create wrappers/plugins for specific frameworks, it should be easy to wrap these utilities.

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628716140

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628716140

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1628699315

Update at https://github.com/cncf/landscape

fulfills cncf/toc#556

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1587848040

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625297845

per cncf/toc#556 and https://github.com/cncf/foundation/blob/master/website-guidelines.md

• If there is a copyright notice at the bottom of the page, copyright should be to the project authors or to CNCF, not the origin company
• include the sentence “We are a Cloud Native Computing Foundation sandbox project.” and the CNCF logo
• include trademark guidelines by either linking to Trademark Usage (directly or via a "Terms of service" page), or by including the following text: "The Linux Foundation® (TLF) has registered trademarks and uses trademarks. For a list of TLF trademarks, see Trademark Usage"

examples:

• https://opentelemetry.io/
• https://backstage.io/

Description of the feature or enhancement:

It would be good if cdk8s had built-in support to import existing values, if exist.

A use case for this would be to instantiate secrets that are then "filled" by init containers or similar: If the secret doesn't exist, create it and set data.mykey to an empty string. However, if it does exist, do not "reset" it, instead import the existing value if data.mykey and re-use that.

In helm this is typically what the lookup function is used for.

Use Case:

See description.
One could of course instantiate a regular kubernetes client "on the side" but that would potentially bloat the code. It would be better (imho) if cdk8s contained clean lookup functions.

Proposed Solution:

Other:

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1625297845

When importing and using CRDs, it may be useful if the manifest will not only include the resource instances but also the CRD itself.

I believe it should be applied in order, before the actual resources are applied. This means that we need to support dependencies/order somehow.

See https://github.com/cdk8s-team/cdk8s-core/actions/runs/1442018769

We propose that cdk8s will have built-in support for kubectl --prune.

Consider the recommended labels