Hello.

Some proto-files have incorrect go_package definition.
Example is udpa/annotations/migrate.proto#L11 that is github.com/cncf/xds/go/annotations while real package is github.com/cncf/xds/go/udpa/annotations

Because the same message is defined in two repos, we cannot run any application relying on this repo. We will have imports of https://github.com/cncf/udpa a long time due to transitive dependency, so we are a bit stuck.

I am investigating how we can mitigate this on our side and will update this if I find anything so other folks that run into this can copy the steps.

this repo doesnt seem to trigger CI on pull requests

im aware that in the older udpa repo there was (sometimes?) a need to recompile the go binary which might not get caught without CI

ref #3

We are just adding this to Envoy, initially just to stop unused imports, but i think many of the other checks can also be useful

While looking at the generated *.pb.go files in this repository, I figured that the generated code had been generated by an older version of the tool which generates Go bindings for grpc services. The latest version can be found here.

Specifically, I'm talking about the generated functions which are used to register service handlers. For example, this one:

As one can see, this function takes two inputs:

• the first one being a *grpc.Server, and
• the second one being the service handler which is to be registered.

Newer versions of the tool generate code which uses the ServiceRegistrar interface as its first input instead. For example, see here. This is more flexible because it allows concrete types than a *grpc.Server on which the service handler is to be registered.

grpc-go provides a version of a gRPC server which uses xDS to get its configuration. You can find that here. For the grpc services contained in *pb.go files in this repository to be registered on the above mentioned xDS-enabled gRPC server, they need to be generated using newer versions of the protoc-gen-go-grpc.

Please upgrade the go/go.mod google.golang.org/grpc because I have found, using go mod graph, that this repository was the reason I got the following error because it references v1.25.1.
cloud.google.com/go/compute/metadata: ambiguous import: found package cloud.google.com/go/compute/metadata in multiple modules:

Other modules have begun to reference the cloud.google.com/go/compute/metadata package. This package was moved out of cloud.google.com/go. Updating the grpc reference should begin to clear up this issue.

My go mod graph produced the following.
github.com/cncf/xds/[email protected] google.golang.org/[email protected]
google.golang.org/[email protected] github.com/envoyproxy/[email protected]
github.com/envoyproxy/[email protected] google.golang.org/[email protected]
google.golang.org/[email protected] cloud.google.com/[email protected]

There are several use cases in Envoy when we need to iterate over a list of items and match each item individually:

• selecting the first ALPN for which a nested filter chain matcher accepts;
• matching an item in DNS SAN list in a TLS cert;
• matching an IP value in XFF header.

Ideally, we want to support all existing matchers for these use cases, instead of adapting all matchers to expect a list.

How should we define this in xds?

Option 1: Add a common input that takes another input and produces an iterator of data items. This requires some changes in the implementations to return an iterator for data inputs.
Option 2: Add a custom matcher that takes another matcher as input. The nested matcher can be any of string, custom input, prefix map, exact map, or custom sub-linear matcher. The custom matcher parses the input and then attempts to match elements one by one.

Ref: envoyproxy/envoy#20666

Currently, proto targets for different languages are defined in a BUILD file by loading the xds_proto_package macro (eg. in //xds/data/orca/v3). The bzl file that defines xds_proto_package again loads macros from rules_go:

This is a bad practice because it forces any other project (see grpc) which depends on any target in //xds/data/orca/v3 to fetch rules_go, even though the project doesn't depend on the go_proto_library target. This is a common pitfall with Bazel, see bazelbuild/bazel#12835

A better package structure is to split targets into different packages by language, eg

• proto_library under //xds/data/orca/v3
• go_proto_library under //xds/data/orca/v3/go
• py_proto_library under //xds/data/orca/v3/py
• cc_proto_library under //xds/data/orca/v3/cc

so that dependencies for specific languages are not fetched when the corresponding package is not needed.

To maintain backwards compatibility, you can still have alias targets in //xds/data/orca/v3that points to other language specific packages.

After improt github.com/cncf/xds/go/xds/type/v3 , go get command get an error

go: main imports
        github.com/cncf/xds/go/xds/type/v3 imports
        cel.dev/expr: cannot find module providing package cel.dev/expr

main.go

package main

import (
        "github.com/cncf/xds/go/xds/type/v3"
)

go.mod

module main

go 1.21

we have just updated the Envoy repo to make use of precompiled protoc which speeds a load of things up

recompiling the go protos i immediately noticed that i was compiling protoc

one option is using protobuf from rules_proto which makes use of the precompiled bins but this then ties you to their version of protobuf

another option is to patch protobuf, this is what Envoy does

In the README.md, I cannot access @cncf/xds-wg, seems the link does not existed.

gRPC recently found out that cncf/xds repo won't work with the HEAD version of protobuf because protobuf:py_proto_library is removed (technically renamed) by protocolbuffers/protobuf#10132. Therefore, build ends up with the following error.

ERROR: Traceback (most recent call last):
        File "/usr/local/google/home/veblush/.cache/bazel/_bazel_veblush/c4652c20fd8d5880d194bf82693e4fee/external/com_github_cncf_udpa/bazel/api_build_system.bzl", line 2, column 66, in <toplevel>
                load("@com_google_protobuf//:protobuf.bzl", _py_proto_library = "py_proto_library")
Error: file '@com_google_protobuf//:protobuf.bzl' does not contain symbol 'py_proto_library'
ERROR: /usr/local/google/home/veblush/git/grpc/BUILD:7688:34: error loading package '@com_github_cncf_udpa//xds/type/v3': Extension file 'bazel/api_build_system.bzl' has errors and referenced by '//:xds_type_upbdefs'

They say that protobuf:py_proto_library is an internal rule so you may want to move away from it.

Anyone writing a Go application that wishes to use Envoy protos will need a stable release of this repo. In case you are not aware, proto definitions cannot be generated independently by libraries, since multiple versions of the generated code for a single message that make their way into the same binary will cause a panic at init time. And without a stability guarantee, use of this repo as the official source of the protos is risky. Please create a stable v1.x release of this repo ASAP. Thank you.

Related to cncf/udpa#42 and envoyproxy/go-control-plane#391.

cc @htuch

I want to create a data-plane xDS compatible from scratch. How should I proceed?

• has this repository already the minimal API-set to bootstrap a proof-of-concecpt?
• the poc will be in Java, how cumbersome it is to build this xDS api using Java -- I know nothing about Starklark
• after built, how can I use this API with a data-plane, let say go-control-plane or Istio, what is the trick to make this working?
• is there any documentation/example/e2e-testing for this API in addition to Envoy website?

following up on slack conversation about possibility of using buf breaking change detection

the context was a discussion about moving Envoy's protos to the xds repo and whether we would be able to avoid running all of Envoy's CI in order to detect breaking changes

i dont have a concrete answer to that yet, im just getting a feel of how buf works, but i think probably the best first step is to add the buf dependency, and some initial linting (#45 )

from there i would suggest we look at if/how it could be used with existing xds protos

my initial (limted) understanding is that it works by checking local protos against published ones - it may not do what we need in terms of above goals - but is probably still quite useful even if it doesnt

For context, see discussion in https://github.com/envoyproxy/envoy/pull/14399/files#r542738561.

I found that go packages, say github.com/cncf/xds/go/udpa/annotations, are placed in submodule github.com/cncf/xds/go.

However, it seems that module github.com/cncf/xds/go is not tagged. According to Go Modules wiki, submodule should be tagged like relative-path-to-root/vX.X.X.
At now, when trying to import package github.com/cncf/xds/go/udpa/annotations, downstream would depends on pseudo-version of module github.com/cncf/xds/go.

github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe

I think it is not very readable and difficult to upgrade. This is not conductive to version control either.
So, I propose whether it is possible to tag submodule properly to better support go users. For example, go/v0.0.1, go/v1.0.0etc, so that other project can use tag to import this module in go.mod.

If I use xds at my Java codebase, there is only two options:

1. copy xds *.proto to my codebase, like grpc-java
2. generate java stab and put it to maven repo.

So, I think we could release official maven package? like io.cncf.xds:xds:3.0.0?

Hello, I have a question on a specific aspect of glob collections. In the definition for glob collections, the following is mentioned:

Globs can exist in arbitrary path locations, e.g. xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/*. Multiple globs may be subscribed to in a DeltaDiscoveryRequest.

Does this mean that the following resource:

xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/resourceName

Belongs to each of these collections?

xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/*
xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/*
xdstp://some-authority/envoy.config.listener.v3.Listener/some/*

Or are glob collection URLs / resource URNs meant to be opaque, without actually implying hierarchy?

Envoy is taking a dependency on protos from this repo. That means Envoy docs need references to the protos here in a better form than raw source code.

Hi,

I have a question regarding the use the node-id for envoy when using xDS.
Is it possible to use the same node-id for envoy when using them in a Replicaset. Generally speaking, is it discouraged to use the same node-id for more than one envoy, even if they should have the exact same config?

Thanks!

One of the recent PRs broke go users: #76.
This type of errors should be prevented by adding cd go && go build ./... step to https://github.com/cncf/xds/blob/main/ci/azure-pipelines.yml.

FYI @adisuissa