cncf / xds Goto Github PK
View Code? Open in Web Editor NEWxDS API Working Group
License: Apache License 2.0
xDS API Working Group
License: Apache License 2.0
Hello.
Some proto-files have incorrect go_package definition.
Example is udpa/annotations/migrate.proto#L11 that is github.com/cncf/xds/go/annotations
while real package is github.com/cncf/xds/go/udpa/annotations
gRPC recently found out that cncf/xds repo
won't work with the HEAD version of protobuf because protobuf:py_proto_library
is removed (technically renamed) by protocolbuffers/protobuf#10132. Therefore, build ends up with the following error.
ERROR: Traceback (most recent call last):
File "/usr/local/google/home/veblush/.cache/bazel/_bazel_veblush/c4652c20fd8d5880d194bf82693e4fee/external/com_github_cncf_udpa/bazel/api_build_system.bzl", line 2, column 66, in <toplevel>
load("@com_google_protobuf//:protobuf.bzl", _py_proto_library = "py_proto_library")
Error: file '@com_google_protobuf//:protobuf.bzl' does not contain symbol 'py_proto_library'
ERROR: /usr/local/google/home/veblush/git/grpc/BUILD:7688:34: error loading package '@com_github_cncf_udpa//xds/type/v3': Extension file 'bazel/api_build_system.bzl' has errors and referenced by '//:xds_type_upbdefs'
They say that protobuf:py_proto_library
is an internal rule so you may want to move away from it.
Please upgrade the go/go.mod google.golang.org/grpc because I have found, using go mod graph, that this repository was the reason I got the following error because it references v1.25.1.
cloud.google.com/go/compute/metadata: ambiguous import: found package cloud.google.com/go/compute/metadata in multiple modules:
Other modules have begun to reference the cloud.google.com/go/compute/metadata
package. This package was moved out of cloud.google.com/go
. Updating the grpc reference should begin to clear up this issue.
My go mod graph produced the following.
github.com/cncf/xds/[email protected] google.golang.org/[email protected]
google.golang.org/[email protected] github.com/envoyproxy/[email protected]
github.com/envoyproxy/[email protected] google.golang.org/[email protected]
google.golang.org/[email protected] cloud.google.com/[email protected]
following up on slack conversation about possibility of using buf
breaking change detection
the context was a discussion about moving Envoy's protos to the xds repo and whether we would be able to avoid running all of Envoy's CI in order to detect breaking changes
i dont have a concrete answer to that yet, im just getting a feel of how buf works, but i think probably the best first step is to add the buf dependency, and some initial linting (#45 )
from there i would suggest we look at if/how it could be used with existing xds protos
my initial (limted) understanding is that it works by checking local protos against published ones - it may not do what we need in terms of above goals - but is probably still quite useful even if it doesnt
We are just adding this to Envoy, initially just to stop unused imports, but i think many of the other checks can also be useful
I want to create a data-plane xDS compatible from scratch. How should I proceed?
we have just updated the Envoy repo to make use of precompiled protoc
which speeds a load of things up
recompiling the go protos i immediately noticed that i was compiling protoc
one option is using protobuf from rules_proto
which makes use of the precompiled bins but this then ties you to their version of protobuf
another option is to patch protobuf, this is what Envoy does
There are several use cases in Envoy when we need to iterate over a list of items and match each item individually:
Ideally, we want to support all existing matchers for these use cases, instead of adapting all matchers to expect a list.
How should we define this in xds?
Option 1: Add a common input that takes another input and produces an iterator of data items. This requires some changes in the implementations to return an iterator for data inputs.
Option 2: Add a custom matcher that takes another matcher as input. The nested matcher can be any of string, custom input, prefix map, exact map, or custom sub-linear matcher. The custom matcher parses the input and then attempts to match elements one by one.
In the README.md, I cannot access @cncf/xds-wg, seems the link does not existed.
Anyone writing a Go application that wishes to use Envoy protos will need a stable release of this repo. In case you are not aware, proto definitions cannot be generated independently by libraries, since multiple versions of the generated code for a single message that make their way into the same binary will cause a panic at init time. And without a stability guarantee, use of this repo as the official source of the protos is risky. Please create a stable v1.x release of this repo ASAP. Thank you.
Related to cncf/udpa#42 and envoyproxy/go-control-plane#391.
cc @htuch
While looking at the generated *.pb.go
files in this repository, I figured that the generated code had been generated by an older version of the tool which generates Go bindings for grpc services. The latest version can be found here.
Specifically, I'm talking about the generated functions which are used to register service handlers. For example, this one:
xds/go/udpa/service/orca/v1/orca.pb.go
Line 264 in 1e77728
As one can see, this function takes two inputs:
*grpc.Server
, andNewer versions of the tool generate code which uses the ServiceRegistrar interface as its first input instead. For example, see here. This is more flexible because it allows concrete types than a *grpc.Server
on which the service handler is to be registered.
grpc-go provides a version of a gRPC server which uses xDS to get its configuration. You can find that here. For the grpc services contained in *pb.go
files in this repository to be registered on the above mentioned xDS-enabled gRPC server, they need to be generated using newer versions of the protoc-gen-go-grpc
.
If I use xds at my Java codebase, there is only two options:
So, I think we could release official maven package? like io.cncf.xds:xds:3.0.0
?
One of the recent PRs broke go users: #76.
This type of errors should be prevented by adding cd go && go build ./...
step to https://github.com/cncf/xds/blob/main/ci/azure-pipelines.yml.
FYI @adisuissa
this repo doesnt seem to trigger CI on pull requests
im aware that in the older udpa repo there was (sometimes?) a need to recompile the go binary which might not get caught without CI
ref #3
Currently, proto targets for different languages are defined in a BUILD file by loading the xds_proto_package macro (eg. in //xds/data/orca/v3
). The bzl file that defines xds_proto_package
again loads macros from rules_go:
xds/bazel/api_build_system.bzl
Line 4 in 1e77728
This is a bad practice because it forces any other project (see grpc) which depends on any target in //xds/data/orca/v3
to fetch rules_go, even though the project doesn't depend on the go_proto_library
target. This is a common pitfall with Bazel, see bazelbuild/bazel#12835
A better package structure is to split targets into different packages by language, eg
so that dependencies for specific languages are not fetched when the corresponding package is not needed.
To maintain backwards compatibility, you can still have alias
targets in //xds/data/orca/v3
that points to other language specific packages.
After improt github.com/cncf/xds/go/xds/type/v3
, go get
command get an error
go: main imports
github.com/cncf/xds/go/xds/type/v3 imports
cel.dev/expr: cannot find module providing package cel.dev/expr
main.go
package main
import (
"github.com/cncf/xds/go/xds/type/v3"
)
go.mod
module main
go 1.21
Envoy is taking a dependency on protos from this repo. That means Envoy docs need references to the protos here in a better form than raw source code.
For context, see discussion in https://github.com/envoyproxy/envoy/pull/14399/files#r542738561.
In #89 the cel.dev/expr based annotations were reintroduced. Unfortunately, there still seem to be issues when trying to use Bazel, Go and Gazelle together, and importing this module.
I've recently added a dependency on github.com/cncf/xds/go/xds/type/matcher/v3
and updated my dependency on go-control-plane to pick up a bugfix (envoyproxy/go-control-plane@77feb56 ).
(I'm not sure my new dependency there matters, tbh. It's just very close and maybe adding confusion.)
This update to go-control-plane updated the version to this library, and thus picked up the cel.dev/expr change.
Now, my builds fail with:
ERROR: /private/var/tmp/_bazel_randerson/01ba2733315bf63f62b0e43c3a981a72/external/com_github_cncf_xds_go/xds/type/v3/BUILD.bazel:3:11: no such target '@dev_cel_expr//:expr': target 'expr' not declared in package '' defined by /private/var/tmp/_bazel_randerson/01ba2733315bf63f62b0e43c3a981a72/external/dev_cel_expr/BUILD.bazel (Tip: use query "@dev_cel_expr//:*"
to see all the targets in that package) and referenced by '@com_github_cncf_xds_go//xds/type/v3:type'
The BUILD file there doesn't actually define any go libraries, so this isn't particularly surprising, but I don't think this commit is working as expected right now.
I'm not really sure if the problem is here or in cel.dev/expr, but ... well, debugging 3 layers of modules down is not fun.
Hi,
I have a question regarding the use the node-id for envoy when using xDS.
Is it possible to use the same node-id for envoy when using them in a Replicaset. Generally speaking, is it discouraged to use the same node-id for more than one envoy, even if they should have the exact same config?
Thanks!
I found that go packages, say github.com/cncf/xds/go/udpa/annotations
, are placed in submodule github.com/cncf/xds/go
.
However, it seems that module github.com/cncf/xds/go
is not tagged. According to Go Modules wiki, submodule should be tagged like relative-path-to-root/vX.X.X
.
At now, when trying to import package github.com/cncf/xds/go/udpa/annotations
, downstream would depends on pseudo-version of module github.com/cncf/xds/go
.
github.com/cncf/xds/go v0.0.0-20211001041855-01bcc9b48dfe
I think it is not very readable and difficult to upgrade. This is not conductive to version control either.
So, I propose whether it is possible to tag submodule properly to better support go users. For example, go/v0.0.1
, go/v1.0.0
etc, so that other project can use tag to import this module in go.mod.
Because the same message is defined in two repos, we cannot run any application relying on this repo. We will have imports of https://github.com/cncf/udpa a long time due to transitive dependency, so we are a bit stuck.
I am investigating how we can mitigate this on our side and will update this if I find anything so other folks that run into this can copy the steps.
Hello, I have a question on a specific aspect of glob collections. In the definition for glob collections, the following is mentioned:
Globs can exist in arbitrary path locations, e.g.
xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/*
. Multiple globs may be subscribed to in a DeltaDiscoveryRequest.
Does this mean that the following resource:
xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/resourceName
Belongs to each of these collections?
xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/path/*
xdstp://some-authority/envoy.config.listener.v3.Listener/some/longer/*
xdstp://some-authority/envoy.config.listener.v3.Listener/some/*
Or are glob collection URLs / resource URNs meant to be opaque, without actually implying hierarchy?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.