cello-proj / cello Goto Github PK

We currently override the type specified in git manifests in the service API. This is behavior is not obvious and probably pretty confusing.

We currently have /projects/$project/targets/$target/operations for initiating git based workflows. Should we use a similar model for non-git based workflows as in both cases they operate on a target? For reference, the .../operations endpoint was added well after /workflow.

The ttl of the vault token can vary based on the workflow used.

Argo workflows can have issues with the history of a large number of workflows. Investigate options for options for handling this. Argo workflows does provide a workflow archive feature.

These should either both have name or not.

Users should easily configure the number of tokens the service can create.

The number of times a vault token can be retrieved can vary based on the workflow used.

This should probably be exposed in a couple of ways:

• via a cli flag
• logged upon start

There are placeholder vars here.

Currently if it doesn't exist, we will return a 500. This should probably be a 404 (or 200/204).

role/secret? pre-generated token? both? other?

Support waiting for connections to complete, etc.

This should support pagination. There are also implications with #115.

Appears to be here and here. Example output:

Starting Argo CloudOps Service
ts=2021-10-05T21:47:18.13841984Z level=info message="loading config 'argo-cloudops.yaml'"
ts=2021-10-05T21:47:18.138684419Z level=info message="loading config 'argo-cloudops.yaml' completed"
Enumerating objects: 24, done.
Counting objects:   4% (1/24)
Counting objects:   8% (2/24)
Counting objects:  12% (3/24)
Counting objects:  16% (4/24)
Counting objects:  20% (5/24)
Counting objects:  25% (6/24)
Counting objects:  29% (7/24)
Counting objects:  33% (8/24)
Counting objects:  37% (9/24)
Counting objects:  41% (10/24)
Counting objects:  45% (11/24)
Counting objects:  50% (12/24)
Counting objects:  54% (13/24)
Counting objects:  58% (14/24)
Counting objects:  62% (15/24)
Counting objects:  66% (16/24)
Counting objects:  70% (17/24)
Counting objects:  75% (18/24)
Counting objects:  79% (19/24)
Counting objects:  83% (20/24)
Counting objects:  87% (21/24)
Counting objects:  91% (22/24)
Counting objects:  95% (23/24)
Counting objects: 100% (24/24)
Counting objects: 100% (24/24), done.
Compressing objects:   5% (1/19)
Compressing objects:  10% (2/19)
Compressing objects:  15% (3/19)
Compressing objects:  21% (4/19)
Compressing objects:  26% (5/19)
Compressing objects:  31% (6/19)
Compressing objects:  36% (7/19)
Compressing objects:  42% (8/19)
Compressing objects:  47% (9/19)
Compressing objects:  52% (10/19)
Compressing objects:  57% (11/19)
Compressing objects:  63% (12/19)
Compressing objects:  68% (13/19)
Compressing objects:  73% (14/19)
Compressing objects:  78% (15/19)
Compressing objects:  84% (16/19)
Compressing objects:  89% (17/19)
Compressing objects:  94% (18/19)
Compressing objects: 100% (19/19)
Compressing objects: 100% (19/19), done.
Total 123 (delta 15), reused 12 (delta 5), pack-reused 99
.....

For example, if you don't give a valid container image, logs returns no error. We should respond with an error with details if possible.

They are trying to convey the same thing, but are slightly different.

Hi, so I can see AppeEngine in the https://github.com/argoproj-labs/argo-cloudops/blob/main/go.mod#L131
but I'm not sure if anyone has ever used this without AWS.

Has anyone got any pointers for how to launch a service in GCP, preferably AppEngine?

We don't need the stack trace. Instead, output the error and os.Exit(1)

Currently the Argo CloudOps manifest format allows for customizing arguments and parameters but does not allow for customizing arguments by workflow type (sync vs diff). This is an issue because there are several arguments that are only valid for one or the other. Environment variables also fit into this issue.

One workaround is to use separate manifests for sync and diff.

One alternative could be to support a way to specify arguments and parameters per action separate from common actions. Ideally this would allow for common args/params to be set as well as customizations per action separately to minimize yaml duplication

This will make it consistent with the rest of the service env vars.

Will need to see if this can be easily determined from the argo workflows api

There is a known goroutine leak in the Argo Workflow log streaming method when the cello logs -f CLI command is terminated in the middle of streaming a log. The logs will continue to stream on the server side. Not a major impact as there are no errors occurring when the CLI is terminated, but still warrants a look to avoid any unnecessary processing or tough issues to solve because of leaks.

TBD

- is much more common/familiar for cli flags.