I'm wondering if dogma cli can have --listener option for watch subcommand.
What do you think?

Example usage

$ cat listener.sh
#!/bin/sh

PATH=$1
TYPE=$2
REV=$3
URL=$4
MODIFIED_AT=$5
echo $PATH $REV $TYPE $URL $MODIFIED_AT
cat > $(basename $PATH)
kill -HUP $(pgrep prometheus)

$ dogma watch --listener listener.sh --streaming 'projFoo/repoA/*.yaml'
/foo.yaml 2 TEXT /api/v1/projects/projFoo/repos/repoA/contents/foo.yaml
/bar.yaml 4 TEXT /api/v1/projects/projFoo/repos/repoA/contents/bar.yaml
<Press CTRL+C>

$ ls
foo.yaml bar.yaml

Background

I have configuration files (specifically, prometheus alert rule files) in a CentralDogma repository.
I'd like to fetch and store those config files and keep fetched-files up-to-date via watch API.
In addition, I need to send a SIGNAL to a certain process after fetching the latest content.
e.g. send kill HUP to prometheus process to let it reload config file

Option details

Users can pass a command name to --listener option.
dogma watch command watches a CentralDogma repository and invokes given command with watchResult
parameters when it gets a notification. The user command will get updated content from STDIN and other
metadata from ARGV.

A directory can be downloaded with #49.
It would also be helpful to support uploading a directory.
A recursive download can be used to backup a Central Dogma directory and
the recursive upload can be used to bootstrap a Central Dogma repository.

dogma put projFoo/repoA/ /local/folder -m "Initialize the repository" --recursive

We need to modernize the CLI since we use the first version of urfave/cli.

We need to bump to V2 due to the supports of dependency injection through context and create more testable code

https://github.com/urfave/cli/blob/3d46fd314ead5e1cdc8db5281f60d9149f737c64/context.go#L15

Before:

After:

func getDogmaClientFromCtx(ctx context.Context) *dogma.Client {
	v, ok := ctx.Value("anyvalue").(*dogma.Client)
	if !ok {
		return nil
	}
	return v
}

func (wc *watchCommand) execute(c *cli.Context) error {
	client := getDogmaClientFromCtx(c.Context)
	if client == nil {
		// create new
	}

	// rest of the code
}

I have created the changes on my fork

Hello.

I tried to create a project via dogma cli but it failed with following error.

$ ./dogma -c http://127.0.0.1:36462 new foooo
failed to parse a JSON document: com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field "creator" (class com.linecorp.centraldogma.internal.api.v1.CreateProjectRequest), not marked as ignorable (3 known properties: "owners", "members", "name"])
 at [Source: (String)"{"name":"foooof","creator":{}}
"; line: 1, column: 31] (through reference chain: com.linecorp.centraldogma.internal.api.v1.CreateProjectRequest["creator"]) (status: 400)

My environment:
centraldogma : 0.39.1
centraldogma-go: master branch (c07c29a)

Regards.

Related: line/centraldogma#435

A client sometimes gets different entries from different replicas in the same Central Dogma cluster, mainly due to replication lag, i.e. Even if you learned that the latest revision of a repository in replica A is 42, you may fail to get a file at revision 42 from replica B.

Currently, in such a case, our Go client simply fails with 404, which is confusing to our users.

We could instead keep the latest known revision numbers of the repositories and retry a few more times when it is certain that 404 error was due to the replication lag.

Which follows line/centraldogma#293.

I realize we're currently using 2 modules inside the project:

• go.linecorp.com/centraldogma
• go.linecorp.com/centraldogma/internal/app/dogma

What is the current method if we have changes in go.linecorp.com/centraldogma and want to use it immediately in go.linecorp.com/centraldogma/internal/app/dogma?

Do we still use replace directive?

If so, I'd like to suggest to utilize go.work that has been introduced in go1.19.

What do you think?

Such as we do in https://github.com/line/centraldogma-rs/blob/master/CONTRIBUTING.md

These steps below I copied from tutorial.

Reproduce steps:

start centraldogma with docker command:

$ docker run -p 36462:36462 --name dogma --rm line/centraldogma

Then running with following steps:

$ cat test.sh
#!/bin/sh

conn='http://localhost:36462'

./dogma -c $conn new projFoo
./dogma -c $conn new projFoo/repoA
echo '{"a":"b"}' > a.json
./dogma -c $conn put projFoo/repoA/samples/a.json a.json -m 'Add a.json'
./dogma -c $conn ls
./dogma -c $conn cat projFoo/repoA/samples/a.json
./dogma -c $conn cat --jsonpath '$.a' projFoo/repoA/samples/a.json

Expected result:

The ./dogma -c http://localhost:36462 cat --jsonpath '$.a' in the last step should returns "b" as mentioned in tutorial website.

Actual Result

invalid character 'b' looking for beginning of value

Dogma version:

$ ./dogma
Central Dogma version 0.41.1 (241c5a8)

And found it in current master branch.

• We need to improving documentation with a lot of examples in README, also have sample code folder in repo

Golang supports HTTP/2 from version 1.6, I think it is pretty mature in Golang and we can take advantage of HTTP/2.
While I was figuring out how to adapt it in our client, I found two situations that we have to consider:

• The oauth2 client we use internally does not support HTTP/2 yet.
• It seems like h2c is not supported in the client-side. See this.

For the first one, I think we can create the oauth2 client directly something like:

// The code creating a client in oauth2
return &http.Client{
	Transport: &Transport{
		Base:   internal.ContextClient(ctx).Transport,
		Source: ReuseTokenSource(nil, src),
	},
}
// Create our own using the exported Struct and field.
return &http.Client{
	Transport: &Transport{
		Base:   &http2.Transport{...},
		Source: ReuseTokenSource(nil, src),
	},
}

The problem is the second one. We just cannot replace the internals of APIs for creating a client with HTTP/2 client because there might be a Central Dogma server which does not use TLS.
So I think we have to provide ways to use the HTTP/1 client as well for non-TLS servers.
I am planning to replace the old NewClient* APIs to use HTTP/2 client internally and add new APIs for HTTP/1 non-TLS client.

/cc @linxGnu and @huydx