buildpacks-community / kpack-cli Goto Github PK

Track of work for Design addressing thre issues:

48. BoM issue

49. Diff issue

50. Why issue

Notes doc

• Prepare design plan (and write "user stories" for the scenarios) [PE]
• Compare the current state of pack and kpack [PE]
• Lay them out in a board [PD] and look for the relationships among them [PD]
• Get input from McNew on the scenarios [PD]
• Draft a solution idea for why the build kicked off and whats in the build [SE]
• Gather examples of diffs in other services [SE]
• Draft a solution idea for diff [SE]
• Get feedback from the team on the feasibility of the ideas [SD]
• Get feedback from Madhura, X, Y, Z on whether these solutions satisfy their needs [SD]

Requested by Kathy Wan (VMware Tanzu SE) in a user interview

As an opps person using kp
I want additional information about each kpack image when performing a list command other than just the name of the image
So that I can at a glance have more context about the application that the image name represents

The idea we discussed was turning kp image list into tabular output and provide the user that created the image, and the "kind" of app (programming language)

Additionally, kp could warn you if you need to upgrade/downgrade your version of kp to align with the version of kpack on your cluster. For example, much like the fly sync command for concourse.

When a user makes a mistake and attempts to create a store without a name, the error message is not very helpful.

Perhaps we could use flags to provide the buildpackages so that the name is the only required positional argument.

Input
tbctl store create --default-repository gopivotal-build-service-private.jfrog.io/deletable gcr.io/cf-build-service-public/fixtures/fixture-buildpackage gcr.io/cf-build-service-public/fixtures/fixture-buildpackage2

Output
Store.experimental.kpack.pivotal.io "gcr.io/cf-build-service-public/fixtures/fixture-buildpackage" is invalid: metadata.name: Invalid value: "gcr.io/cf-build-service-public/fixtures/fixture-buildpackage": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is 'a-z0-9?(.a-z0-9?)*')

Can we look into validating (registry and git) secrets at the time of creation? Ref.. kp secret create command.
Currently it will take in whatever you give and will complain only at the time the creds are used - during the build phase, I think.

If required, we can make it optional with a --verify flag

In some ways, perform the same validation that is already done at runtime by kpack, (using https://github.com/google/go-containerregistry) to also apply at the time of secret creation by kp?
Obvious benefits would be early feedback for the user and hence less debug/support requests for us.

kp import is a powerful command capable of making a broad set of resource changes (based on what is defined in the DependencyDescriptor yaml file). Considering this, it would be helpful to provide the user

• a summary visualization of the changes that will be applied
• and request confirmation before proceeding to apply the changes

This is probably more helpful/relevant in the upgrade scenario.

How to display the summary of changes is to be decided - do we want a tabular form or a delta on the yaml? or both forms supported by different flags are all up for discussion...

Prior art includes kapp deploy|delete and the Concourse fly CLI

Combine kp create + update (or is it patch?) commands into an apply or upsert command so that the same command can be used when an image is created as when an update comes and a new version needs to be built

This week I saw thomas risberg from Spring Cloud Data Flow type "kp images list" and it didn't work (although the error message did suggest the correct noun!).

Some other CLIs, ex kubectl, allow the user to type plural or singular nouns and the command works either way. In order to reduce minor friction in DevOps engineers' workflows, and to make the kp cli easier to learn, I propose adding plural aliases to all the kpack noun subcommands

build -> builds
builder -> builders
cluster-builder -> cluster-builders
image -> images
secret -> secrets
stack -> stacks

exceptions:
store
version
help

Similar to: #51

Propose an additional attribute, “Reason-Message” be added to the build object, which can be printed in the context of the existing “Reason” attribute. This “Reason-Details” object would be printed for these two commands:

$ kp build status image-name
$ kp image status image-name

Today both of these commands give a one line Reason for why a build happened, ex.

The format in which it would be printed:

Reason: <Reason-code> - <details of why>

Examples:

Reason: CONFIG - Change to environment variable var=var-value
Reason: BUILDER - Buildpack-name changed to version-number
Reason: STACK - Stack-name changed to version-number

Why?

Currently, the kp cli does not support adding bash tab completion aliases similar to the previous pb cli.

We got feedback that tab completion was very useful in the pb cli.

What

Users should be able to add . $(kp completion) to their bash_profile.

We use a ccb-service-account in the build-service namespace for ccb service account: https://github.com/pivotal/build-service-cli/blob/master/pkg/commands/customclusterbuilder/create.go#L71

This couples the CLI to the build-service namespace. This service account can probably be the canonical.service-account and can be in the kpack namespace.

Command to install and/or update stacks, stores, and custom cluster builders given configuration.

Useful for creating or updating multiple dependencies.

Config File

dependencies.yaml

apiVersion: kp.kpack.io/v1alpha1
kind: DependencyDescriptor
stores:
- name: default
  sources:
  - image: gcr.io/paketo-buildpacks/java@sha256:abcxyz
  - image: gcr.io/paketo-buildpacks/nodejs@sha256:abcxyz
  - image: gcr.io/paketo-buildpacks/go@sha256:abcxyz
stacks:
- name: default
   buildImage:
     image: "gcr.io/paketo-buildpacks/build@sha256:abcxyz"
   runImage:
     image: "gcr.io/paketo-buildpacks/run@sha256:abcxyz"
custom_cluster_builders:
- image: my-registry.io/default
  order:
  - group:
    - id: paketo-buildpacks/java
    - id: paketo-buildpacks/nodejs
    - id: paketo-buildpacks/go

Interface:

kp import --config dependencies.yaml

Prints resources it will create or update
If resource does not exist, create it, otherwise update it.

Notes

• Only supports Stacks, Stores, and Custom Cluster Builders
• stores, stacks, custom_cluster_builders keys all optional. Only update resources provided.

Questions:

• Should users be able/required to provide repositories for stores and stacks?

What

When a --wait command exits unsuccessfully it should accurately describe the error. (Dropped connection, Failed build, Builder not ready, etc)

Why

Currently, an error will just result in an opaque message: Error: update to image failed. This makes debugging the command difficult.

kp create and patch commands should be able to produce yaml output with a --dry-run flag.

Context: this helps to enable a Gitops workflow so that the yaml can be reused in the git repo that triggers the pipeline

[Reported by @mgbrodi]

Problem: when kp image trigger was run, it resulted in an unrelated and confusing error message, especially since the user expects a build to be triggered by the command.

Error: no builds found

The expectation is to show a more helpful error message.

For this particular case, the root cause of failure was identified as invalid creds for the image - git secret for a private repo was created with a public key (instead of a private key) and also there was a basic auth secret created for an MFA enabled Github account

This

kp image create foo foo.io/foo --git https://github.com/odedia/petclinic.git --git-revision master --sub-path spring-petclinic-api-gateway --env BP_MAVEN_BUILD_ARGUMENTS="-Dmaven.test.skip=true -Pk8s package"

returns

Error: env vars are improperly formatted

probably due to this: https://github.com/pivotal/build-service-cli/blob/master/pkg/image/factory.go#L101

Users have reported kp import hanging or taking a long time.

User logs installing on dockerhub:

kp import -f descriptor.yaml
Importing Cluster Store 'default'...
Uploading to 'my-repo'...

Quote from user using harbor:

"Quick update: the kp import command line seems to hang"

User quote:

"Import is underway but is pretty slow, I’m capped at around 5.0 Mbps for the download speed."

Actions:

It's hard to validate this issue is more than just an internet connection bottleneck.

However, a potential improvement would be importing in parallel.

With the added mixin metadata stack status is incredibly verbose and obfuscates what is important. Pack will hide mixin metadata on inspect-builder behind a --verbose, we should do the same.

As an example this is the output from a paketo buildpacks base stack. (Not even full!)

Status: Ready
Id: io.buildpacks.stacks.bionic
Run Image: index.docker.io/kpack/run@sha256:3e94f2a7450111c3f8ebb9d00db5a3c8c024e5436891a83407b53cff994c399d
Build Image: index.docker.io/kpack/build@sha256:4070fb1c95b60678bd85f50a22d41cbef9b684102bb40d3859d6b4736eb896da
Mixins: adduser, apt, base-files, base-passwd, bash, bsdutils, bzip2, ca-certificates, coreutils, dash, debconf, debianutils, diffutils, dpkg, e2fsprogs, fdisk, findutils, gcc-8-base:amd64, gpgv, grep, gzip, hostname, init-system-helpers, libacl1:amd64, libapt-pkg5.0:amd64, libattr1:amd64, libaudit-common, libaudit1:amd64, libblkid1:amd64, libbz2-1.0:amd64, libc-bin, libc6:amd64, libcap-ng0:amd64, libcom-err2:amd64, libdb5.3:amd64, libdebconfclient0:amd64, libext2fs2:amd64, libfdisk1:amd64, libffi6:amd64, libgcc1:amd64, libgcrypt20:amd64, libgmp10:amd64, libgnutls30:amd64, libgpg-error0:amd64, libhogweed4:amd64, libidn2-0:amd64, liblz4-1:amd64, liblzma5:amd64, libmount1:amd64, libncurses5:amd64, libncursesw5:amd64, libnettle6:amd64, libp11-kit0:amd64, libpam-modules-bin, libpam-modules:amd64, libpam-runtime, libpam0g:amd64, libpcre3:amd64, libprocps6:amd64, libseccomp2:amd64, libselinux1:amd64, libsemanage-common, libsemanage1:amd64, libsepol1:amd64, libsmartcols1:amd64, libss2:amd64, libssl1.1:amd64, libstdc++6:amd64, libsystemd0:amd64, libtasn1-6:amd64, libtinfo5:amd64, libudev1:amd64, libunistring2:amd64, libuuid1:amd64, libzstd1:amd64, login, lsb-base, mawk, mount, ncurses-base, ncurses-bin, openssl, passwd, perl-base, procps, sed, sensible-utils, sysvinit-utils, tar, ubuntu-keyring, util-linux, zlib1g:amd64, build:adduser, build:apt, build:base-files, build:base-passwd, build:bash, build:bsdutils, build:bzip2, build:ca-certificates, build:coreutils, build:dash, build:debconf, build:debianutils, build:diffutils, build:dpkg, build:e2fsprogs, build:fdisk, build:findutils, build:gcc-8-base:amd64, build:gpgv, build:grep, build:gzip, build:hostname, build:init-system-helpers, build:libacl1:amd64, build:libapt-pkg5.0:amd64, build:libattr1:amd64, build:libaudit-common, build:libaudit1:amd64, build:libblkid1:amd64, build:libbz2-1.0:amd64, build:libc-bin, build:libc6:amd64, build:libcap-ng0:amd64, build:libcom-err2:amd64, build:libdb5.3:amd64, build:libdebconfclient0:amd64, build:libext2fs2:amd64, build:libfdisk1:amd64, build:libffi6:amd64, build:libgcc1:amd64, build:libgcrypt20:amd64, build:libgmp10:amd64, build:libgnutls30:amd64, build:libgpg-error0:amd64, build:libhogweed4:amd64, build:libidn2-0:amd64, build:liblz4-1:amd64, build:liblzma5:amd64, build:libmount1:amd64, build:libncurses5:amd64, build:libncursesw5:amd64, build:libnettle6:amd64, build:libp11-kit0:amd64, build:libpam-modules-bin, build:libpam-modules:amd64, build:libpam-runtime, build:libpam0g:amd64, build:libpcre3:amd64, build:libprocps6:amd64, build:libseccomp2:amd64, build:libselinux1:amd64, build:libsemanage-common, build:libsemanage1:amd64, build:libsepol1:amd64, build:libsmartcols1:amd64, build:libss2:amd64, build:libssl1.1:amd64, build:libstdc++6:amd64, build:libsystemd0:amd64, build:libtasn1-6:amd64, build:libtinfo5:amd64, build:libudev1:amd64, build:libunistring2:amd64, build:libuuid1:amd64, build:libzstd1:amd64, build:login, build:lsb-base, build:mawk, build:mount, build:ncurses-base, build:ncurses-bin, build:openssl, build:passwd, build:perl-base, build:procps, build:sed, build:sensible-utils, build:sysvinit-utils, build:tar, build:ubuntu-keyring, build:util-linux, build:zlib1g:amd64, build:binutils, build:binutils-common:amd64, build:binutils-x86-64-linux-gnu, build:build-essential, build:cpp, build:cpp-7, build:curl, build:dirmngr, build:dpkg-dev, build:fakeroot, build:g++, build:g++-7, build:gcc, build:gcc-7, build:gcc-7-base:amd64, build:git, build:git-man, build:gnupg, build:gnupg-l10n, build:gnupg-utils, build:gpg, build:gpg-agent, build:gpg-wks-client, build:gpg-wks-server, build:gpgconf, build:gpgsm, build:krb5-locales, build:less, build:libalgorithm-diff-perl, build:libalgorithm-diff-xs-perl, build:libalgorithm-merge-perl, build:libasan4:amd64, build:libasn1-8-heimdal:amd64, build:libassuan0:amd64, build:libatomic1:amd64, build:libbinutils:amd64, build:libbsd0:amd64, build:libc-dev-bin, build:libc6-dev:amd64, build:libcc1-0:amd64, build:libcilkrts5:amd64, build:libcurl3-gnutls:amd64, build:libcurl4:amd64, build:libdpkg-perl, build:libedit2:amd64, build:liberror-perl, build:libexpat1:amd64, build:libfakeroot:amd64, build:libfile-fcntllock-perl, build:libgcc-7-dev:amd64, build:libgdbm-compat4:amd64, build:libgdbm5:amd64, build:libgomp1:amd64, build:libgssapi-krb5-2:amd64, build:libgssapi3-heimdal:amd64, build:libhcrypto4-heimdal:amd64, build:libheimbase1-heimdal:amd64, build:libheimntlm0-heimdal:amd64, build:libhx509-5-heimdal:amd64, build:libisl19:amd64, build:libitm1:amd64, build:libk5crypto3:amd64, build:libkeyutils1:amd64, build:libkrb5-26-heimdal:amd64, build:libkrb5-3:amd64, build:libkrb5support0:amd64, build:libksba8:amd64, build:libldap-2.4-2:amd64, build:libldap-common, build:liblocale-gettext-perl, build:liblsan0:amd64, build:libmpc3:amd64, build:libmpfr6:amd64, build:libmpx2:amd64, build:libnghttp2-14:amd64, build:libnpth0:amd64, build:libperl5.26:amd64, build:libpsl5:amd64, build:libquadmath0:amd64, build:libreadline7:amd64, build:libroken18-heimdal:amd64, build:librtmp1:amd64, build:libsasl2-2:amd64, build:libsasl2-modules-db:amd64, build:libsasl2-modules:amd64, build:libsqlite3-0:amd64, build:libssl1.0.0:amd64, build:libstdc++-7-dev:amd64, build:libtsan0:amd64, build:libubsan0:amd64, build:libwind0-heimdal:amd64, build:libx11-6:amd64, build:libx11-data, build:libxau6:amd64, build:libxcb1:amd64, build:libxdmcp6:amd64, build:libxext6:amd64, build:libxmuu1:amd64, build:linux-libc-dev:amd64, build:make, build:manpages, build:manpages-dev, build:multiarch-support, build:netbase, build:openssh-client, build:patch, build:perl, build:perl-modules-5.26, build:pinentry-curses, build:publicsuffix, build:readline-common, build:xauth, build:xz-utils, run:adduser, run:apt, run:base-files, run:base-passwd, run:bash, run:bsdutils, run:bzip2, run:ca-certificates, run:coreutils, run:dash, run:debconf, run:debianutils, run:diffutils, run:dpkg, run:e2fsprogs, run:fdisk, run:findutils, run:gcc-8-base:amd64, run:gpgv, run:grep, run:gzip, run:hostname, run:init-system-helpers, run:libacl1:amd64, run:libapt-pkg5.0:amd64, run:libattr1:amd64, run:libaudit-common, run:libaudit1:amd64, run:libblkid1:amd64, run:libbz2-1.0:amd64, run:libc-bin, run:libc6:amd64, run:libcap-ng0:amd64, run:libcom-err2:amd64, run:libdb5.3:amd64, run:libdebconfclient0:amd64, run:libext2fs2:amd64, run:libfdisk1:amd64, run:libffi6:amd64, run:libgcc1:amd64, run:libgcrypt20:amd64, run:libgmp10:amd64, run:libgnutls30:amd64, run:libgpg-error0:amd64, run:libhogweed4:amd64, run:libidn2-0:amd64, run:liblz4-1:amd64, run:liblzma5:amd64, run:libmount1:amd64, run:libncurses5:amd64, run:libncursesw5:amd64, run:libnettle6:amd64, run:libp11-kit0:amd64, run:libpam-modules-bin, run:libpam-modules:amd64, run:libpam-runtime, run:libpam0g:amd64, run:libpcre3:amd64, run:libprocps6:amd64, run:libseccomp2:amd64, run:libselinux1:amd64, run:libsemanage-common, run:libsemanage1:amd64, run:libsepol1:amd64, run:libsmartcols1:amd64, run:libss2:amd64, run:libssl1.1:amd64, run:libstdc++6:amd64, run:libsystemd0:amd64, run:libtasn1-6:amd64, run:libtinfo5:amd64, run:libudev1:amd64, run:libunistring2:amd64, run:libuuid1:amd64, run:libzstd1:amd64, run:login, run:lsb-base, run:mawk, run:mount, run:ncurses-base, run:ncurses-bin, run:openssl, run:passwd, run:perl-base, run:procps, run:sed, run:sensible-utils, run:sysvinit-utils, run:tar, run:ubuntu-keyring, run:util-linux, run:zlib1g:amd64

Problem Currently, there is not an easy way to detect or correlate if an update/create of an image results in a successful build. This makes it difficult to integrate in a ci pipeline where you want to know if the update "worked" and easily see build logs.

Proposal: Add --wait to image create and image patch. This flag will tail related logs and exit with the correct exit status when the resulting build either succeeds of fails.

Open Question: Is --wait the right name for this?

Outcome

For commands that require it, kp cli tries to read a ConfigMap named kp-config from the kpack namespace.
Using this ConfigMap, kp can fill in values for commands that need:

• The repository used to relocate stack, store, and custom builder images
• a ServiceAccount to attach to resources
• Any future state the cli may want to get

ConfigMap schema:

apiVersion: v1
kind: ConfigMap
metadata:
  name:  kp-config
  namespace: kpack
data:
  canonical.repository:
  canonical.repository.serviceaccount:

$ kp import -f file.yml
> ...
Importing Cluster Stack 'tiny'...
panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/pivotal/build-service-cli/pkg/commands/import.importHelper.ImportStacks(0xc0006ce1c0, 0x14, 0xc0006ce460, 0x14, 0xc00019694c, 0x4, 0xc000196948, 0x4, 0xc000358be0, 0x1, ...)
	/Users/pivotal/workspace/build-service-cli/pkg/commands/import/import.go:223 +0x3f1

This could happen with a kp X create or a kubectl apply -f resource.yml causing kp import to fail

Similar to: #51

Problem

While kp build status is certainly helpful, we currently do not provide all the Bill of Materials metadata the lifecycle can make available to the kpack CLI from upstream. Doing so would be particularly helpful to provide information about the dependencies shipped by the buildpacks used to execute the build. This information would be helpful to users when performing audit functions and determining which image to promote to a new environment.

Outcomes

Given I have created the image config node-test-app
When I run kp build status node node-test-app --bom
Then I see lifecycle BOM output

{"remote":null,"local":[{"name":"node","version":"10.22.0","metadata":{"licenses":[],"name":"Node Engine","sha256":"43616969dd39d52c9d3c0a4ed5e66600133356877ab8344a916638e7f5794490","stacks":["io.buildpacks.stacks.bionic","org.cloudfoundry.stacks.cflinuxfs3"],"uri":"https://buildpacks.cloudfoundry.org/dependencies/node/node_10.22.0_linux_x64_cflinuxfs3_43616969.tgz"},"buildpack":{"id":"paketo-buildpacks/node-engine","version":"0.0.260"}},{"name":"node_modules","version":"","metadata":null,"buildpack":{"id":"paketo-buildpacks/npm","version":"0.1.79"}}]}

Notes

• This story is intended to surface the bom metadata in the kp CLI and does not preclude further structuring to make it more legible to users
• Usage of common pretty print tools like jq improve the legibility of the metadata dramatically, could it be a quick win to encourage people to download it in our docs or in the CLI help output (or both?)

Problem Currently, tbctl image create creates an image configuration without a cache size. This precludes caching from being utilized across builds which will significantly impact build speed.

Proposal We default an initial cache to images created with image create.

Questions

• What cache size should we default? The pb cli defaulted 2Gi.
• Should we allow you to override the cache size?
• Should we not default if the cluster does not have a default storage class?
• Should this be done in the kpack webhook?

When an image does not have any environment variables configured, running: tbctl image patch some-image -e ANY=THING

Will result in:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1e2c553]

goroutine 1 [running]:
github.com/pivotal/build-service-cli/pkg/image.(*PatchFactory).setBuild(0xc0000fe630, 0xc0000d4500, 0x0, 0x0)
        /Users/matthewmcnew/workspace/build-service-cli/pkg/image/patch_factory.go:172 +0x283
github.com/pivotal/build-service-cli/pkg/image.(*PatchFactory).MakePatch(0xc0000fe630, 0xc0002d8780, 0x8, 0xc0003c81e0, 0x0, 0x0, 0x0)
        /Users/matthewmcnew/workspace/build-service-cli/pkg/image/patch_factory.go:41 +0xbc
github.com/pivotal/build-service-cli/pkg/commands/image.NewPatchCommand.func1(0xc0002bbb80, 0xc0003833b0, 0x1, 0x5, 0x0, 0x0)
        /Users/matthewmcnew/workspace/build-service-cli/pkg/commands/image/patch.go:66 +0x1e0
github.com/spf13/cobra.(*Command).execute(0xc0002bbb80, 0xc000383360, 0x5, 0x5, 0xc0002bbb80, 0xc000383360)
        /Users/matthewmcnew/go/pkg/mod/github.com/spf13/[email protected]/command.go:838 +0x453
github.com/spf13/cobra.(*Command).ExecuteC(0xc0002bb080, 0xc0001a7f38, 0x8, 0x8)
        /Users/matthewmcnew/go/pkg/mod/github.com/spf13/[email protected]/command.go:943 +0x317
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/matthewmcnew/go/pkg/mod/github.com/spf13/[email protected]/command.go:883
main.main()
        /Users/matthewmcnew/workspace/build-service-cli/cmd/tbctl/main.go:53 +0x430

It apparently is running into a nil pointer dereference on the spec.build. The patch logic should support a nil spec.build.

Requested by James Musselwhite (VMware Tanzu SE) in a user interview.

Use Case:

As a developer or platform operator initiating an action that I know will result in a new build associated with a given image
I want to initiate a log streaming command prior to the build being kicked off
So I can monitor the build's progress
Or have a separate terminal window always waiting for logs to be streamed for a kpack image I care about

--git is confusing, I thought it was a boolean flag signaling this is a git secret.

If I try to update a stack with a new build or run image but not update the other, the stack will not be updated:

$ kp stack status default
> Status:         Ready
Id:             org.cloudfoundry.stacks.cflinuxfs3
Run Image:      registry.io/run@sha256:abc
Build Image:      registry.io/build@sha256:xyz

$ kp stack update stack default -r registry.io/run@sha256:def -b  registry.io/build@sha256:xyz
Build and Run images already exist in stack
Stack Unchanged

The use case could be trying to patch a dependency that is only in one of your two stack images.

Requested by Jason Morgan (VMware Tanzu SE)

As a user of kp
I want to be able to more granularly filter what is returned to me in the builds table based on attributes other than the kpack image they are associated with
So I can gain insight into cluster wide patterns

Jason particularly requested filtering the builds table for all builds based on the following attributes:

• Success/Failure
• Any one of the rebuild reasons

He also went on to discuss being shown images that use a particular builder, clusterstack, or clusterstore.

Although the request was framed as a change to the builds table, seems what he is really after is a flexible querying interface to be addressed in kp.

Some changes to take note of:

• Add Status Reason and Status Message when they are set on the Build's Status
• Rename Reason to Build Reason
• Show Pod Name
• Do not show current container as I felt like it was not as useful.

Before:

After with Build Status Reason + Message:

After with no Build Status Reason + Message:

#4 added support for --wait on image patches and creates. This allows users to follow the result of an kp action without executing build logs <image-name> and allows it to be used naturally in a ci automation context.

However, without --wait support on a trigger a user will not be able to use the same technique with an image trigger.

Similar, to kp image patch it is especially useful as part of automation to ensure that updates to kpack builders are successful by waiting for them to succeed.

Following #29

Context:

Some commands require a tag as a positional argument:

kp custom-builder create <name> <tag> [flags]
kp custom-cluster-builder create <name> <tag> [flags]
kp image create <name> <tag> [flags]

and some commands require a --defaultRepository flag argument:

kp stack create my-stack --default-repository some-registry.io/some-repo --build-image my-registry.com/build --run-image my-registry.com/run
kp store create my-store my-registry.com/my-buildpackage --default-repository some-registry.io/some-repo

Motivations

With the introduction of kp-config here, kp is able to determine the canonicalrepository and canonicalrepository-secret of the kpack installation. This is useful because this means that the installation already has a repository that kpack can pull from and kp can use this repository for the creation of any non-namespaced resources instead of requiring it as an input to commands via tag or --defaultRepository.

The use of multiple positional args in a command can be confusing and the use of flag arguments beyond a primary position arg (eg. name) can make writing commands easier (order doesn't matter, flags are more explicit, etc). Since we are looking to remove repositories from some commands, it would be a good time to change the command interfaces to use flag args.

Outcomes

1. Commands that use a positional <tag> argument should use a --tag arg in all cases

kp custom-builder create <name> <tag> [flags] -> kp custom-builder create <name> --tag <tag> [flags]
kp custom-cluster-builder create <name> <tag> [flags] -> kp custom-cluster-builder create <name> --tag <tag> [flags]
kp image create <name> <tag> [flags] -> kp image create <name> --tag <tag> [flags]

2. kp ccb create can default the tag to <canonical-repo>/ccb-<name>

This tag should still be able to be set via --tag but defaulting to the canonical repo is a nice experience. Keeping --tag keeps kp cb/ccb create a consistent interface, and allows for more flexibility. Appending ccb-<name> should make the tag unique. If the --tag is not set and canonical-repo is not set, the command should error.

3. Commands that use --defaultRepository should not accept a --defaultRepository argument and should assume the canonical-repository. If the canonical-repository is not set, commands should error.

kp stack create my-stack --default-repository some-registry.io/some-repo --build-image my-registry.com/build --run-image my-registry.com/run -> kp stack create my-stack --build-image my-registry.com/build --run-image my-registry.com/run
kp store create my-store my-registry.com/my-buildpackage --default-repository some-registry.io/some-repo -> kp store create my-store my-registry.com/my-buildpackage

4. Store commands use flag arguments for buildpackages.

Since we are removing multiple-positional args from other commands, it's a good time to remove the buildpackage position args in favor of variable flag args:

kp store create my-store my-registry.com/my-buildpackage my-registry.com/my-buildpackage2 

to

kp store create my-store --buildpackage my-registry.com/my-buildpackage -b my-registry.com/my-buildpackage2

Note: --buildpackage is new flag -b for short

kp build status has a line in its output for "Reasons" whereas kp image status has a line for "Reason," both of which are associated with the same key value pair (to explain the reason a build kicked off).

Let's change one of them so that they are both consistent.

I propose changing kp build status to say Reason instead of Reasons because even when there are multiple reasons, each will show up on a new line, making the user read each reason separately. Also Reason feels like it makes sense for singular and plural values whereas Reasons feels like it only makes sense for plural values.

Current state for kp cli version 0.0.1-build.3 3e7d6d3:

[Reported by @mgbrodi]

Problem: when creating a secret for a private repo using the ssh key, kpack cli does not validate if the provided key is a valid private key. Example:

kp secret create my-git-ssh-cred --git-url [email protected] --git-ssh-key /path/to/git/ssh-private-key.pem

Providing a public key for the --git-ssh-key flag does not result in any error requiring a private key (and not the public key).

Solution: it would be ideal if we could catch this at an earlier stage, instead of time spend debugging during an image build/trigger. In addition, help text could be updated to emphasize private key. Snippet of current help text:

...
  "--git-url" and "--git-ssh-key" to create SSH based git credentials.
  "--git-url" should not contain the repository path (eg. [email protected] not [email protected]:my/repo)
  Alternatively, provided the credentials in the "GIT_SSH_KEY_PATH" env var instead of the "--git-ssh-key" flag.
...

Related issue: validate secrets at the time of creation #22

Requested by James Musselwhite in a user interview:

As a platform/apps operator using kp
I want to be alerted as to which images will be rebuilt before executing a command that updates a clusterstack, clusterstore, or builder resources
So that I can confirm that this outcome is desired

In the output of the kp build status command, the sixth item is "Source" which refers to the origin of the app code. Since some Java developers are put off by the prospect of building from source using kpack, and want to use pre-built artifacts like JARs or WARs, I think we should change this key name to be something more inclusive.

Ideas:

• App Code
• App Origin
• Application
• App Input

Problem

Today when you type a noun subcommand (ex. kp image), the help text for that command prints by default, but when you add a verb onto that noun subcommand (ex. kp image status), an error prints.

Example:

Instead of printing an error when an argument is missing, we should print the help text associated with that verb subcommand, as if the user had typed the -h flag.

We can still print an error (ex. Error: accepts 1 arg(s), received 0), but we should also print the help text and state WHICH arg is missing.

Example:

Mockup of potential solution:

When a verb subcommand does not have the necessary args, help text and an error message are printed together.

$kp image status
Error: accepts 1 arg(s) - <name>, received 0

`kp image status` prints detailed information about the status of a specific image in the provided namespace.

namespace defaults to the kubernetes current-context namespace.

Usage:
  kp image status <name> [flags]

Examples:
kp image status my-image
kp image status my-other-image -n my-namespace

Flags:
  -h, --help               help for status
  -n, --namespace string   kubernetes namespace

Alternative solution:

Provide a human readable error message that explains why the error happened and what you can do about it. For example:

$ kubectl describe
error: You must specify the type of resource to describe. Use "kubectl api-resources" for a complete list of supported resources.

Currently, running kp import will exit successfully even if the resulting resources do not reconcile successfully. This prevents kp import from failing an automated import pipeline job.

registry relocation commands:
kp import
kp clusterstack create
kp clusterstore create

Leads to problems rendering well in a smaller terminal window. we could have a verbose flag that prints more or do some kind of "detect length" before printing output

The output of custom-builder status almost completely matches the output of pack inspect-builder. However, it is missing the homepage column. To actually find the homepage (and documentation) for buildpack currently, users need to dig into the store status which currently only displays the homepage of the buildpackage not the individual buildpacks.

Problem Currently, the kp cli will output the builds table with the fully qualified image reference. This is the most valuable information about the build but, will wrap on small screens.

Proposal We should only show output when it will properly be displayed on the terminal output,.

This was requested from engineers at Citi who were using the kp image create --wait source upload feature because the blank output did not provide them with confidence the the command was actually executing.

What

kp build logs should support a --timestamp flag that will prefix log lines with timestamp. This is equivalent to kubectl logs timestamp.

Why

Timestamps will be helpful to diagnose long builds and isolate unexpected behavior.