mbobakov / grpc-consul-resolver Goto Github PK

That will also require to update populateEndpoints func:
clientConn.NewAddress(conns) -> clientConn.UpdateState(resolver.State{Addresses: conns})

Should move after if err != nil

Even though consul returns services sorted by "near" property it all gets overridden with this line.

It doesn't affect application when one uses round robin balancer, but with pick_first this bug makes it impossible to use any type of sorting.

hello, I have a question: Consul starts ACL and passes token.

conn, err = grpc.Dial(
    fmt.Sprintf("consul://%s:%d/%s?token=%s&wait=14s&tag=%s", consulHost, consulPort, srvName, token, srvTag),
    grpc.WithTransportCredentials(insecure.NewCredentials()),
    grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy": "round_robin"}`),
)

error message：
"error": "rpc error: code = Unavailable desc = last resolver error: produced zero addresses"

Thanks!

Does this item only support one label for filtering? I want to use multiple tags to select services (for example: tags [] string), just like the official ServiceMultipleTags() method, but this project does not seem to support it.

Background

My project serves as a client resolver for Consul, fetching endpoints and providing them to gRPC clients. To enhance the flexibility of configurations, I propose adding support for gRPC service config storage in the Consul KV store.

Feature Request

I want to store gRPC service configurations as keys in the Consul KV store. This will enable users to define service-specific gRPC settings centrally in Consul, improving manageability and reconfigurability.

Proposed Implementation

ConsulKV Store Integration:

Integrate the ConsulKV store functionality to read gRPC service configurations.
Allow users to define gRPC service config as JSON in Consul.

Path to the service config could be provided as URL parameter: Link

In case of failed config parsing, we could throw the log error and continue to work or throw the error and exit. WDYT?

Additional Information

Link to gRPC service config documentation
Example in DNS resolver. They lookup for TXT record

Acceptance Criteria:

• The resolver successfully fetches and applies gRPC service config from the Consul KV store.
• The test covers the usecase of gRPC service config fetch
• Documentation is updated to reflect the new feature.

for _, s := range ss {
address := s.Service.Address
if s.Service.Address == "" {
address = s.Node.Address
}
ee = append(ee, fmt.Sprintf("%s:%d", address, s.Service.Port))
}

thx

in grpc

Our automated dependency managers are still running into issues from this package due to tooling previously added to the go.mod which has been since fixed.

Could we please have a release made of those changes such that we do not pin on a commit sha?

go/pkg/mod/github.com/mbobakov/[email protected]/builder.go:20:65: cannot use url.Endpoint (value of type func() string) as type string in array or slice literal

build.go, line19
dsn := strings.Join([]string{schemeName + ":/", url.Authority, url.Endpoint}, "/")

This will ignore all parameters.

Changed to dsn := url.URL.String() makes parameter works.

Hey,
I read the source code and find that already full implement for grpc.Resolver and adaptor the consul api, the functionality is ok.
But i doubt this library maybe not work smoothly in production envirnments.
According to the example in the document, one connection created is ok, But if i want
to create many connections in my concurrency business scenarios like:

conn1, err := grpc.Dial(
        "consul://127.0.0.1:8500/whoami?wait=14s&tag=manual",
     )
....
conn2, err := grpc.Dial(
        "consul://127.0.0.1:8500/whoami?wait=14s&tag=manual",
     )

Above codes call two times grpc.Builder inter which be implemented in your code. This will lead twice call of under codes:

	go watchConsulService(ctx, cli.Health(), tgt, pipe)
	go populateEndpoints(ctx, cc, pipe)

Is this suitable or some logics that i am misunderstanding ? And how to use this libarary in correct way ?

In the function watchConsulService there's an endless loop and in line 51 we have:

if err != nil {
	grpclog.Errorf("[Consul resolver] Couldn't fetch endpoints. target={%s}", tgt.String())
	continue
}

The problem now is that when this error occurs (maybe the endpoint is not currently registered in consul) then we keep hammering consul endlessly in a very fast rate (plus fill the log with lines such as demonstrated below).

I suggest to add a delay between recurring attempts, e.g. exponential backoff or similar.

Some log lines for example:

ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
ERROR: 2019/05/15 05:35:52 [Consul resolver] Couldn't fetch endpoints. target={service='santa-attrendpoint' healthy='true' tag=''}
.
.
. 
and so on... 

Service query loop in watchConsulService is not terminated on ctx.Done().

The fix is shown in A Tour of Go: Concurrency.

res := make(chan []string)
quit := make(chan struct{})

		select {
		case res <- ee:
			continue
		case <-quit:
			return
		}
	}
}()
  
for {
	select {
	case ee := <-res:
		out <- ee
	case <-ctx.Done():
		close(quit)
		return
	}
}

Hi,help wanted!
I user grpc-consul-resolver as below:

package initialize

import (
	"fmt"

	_ "github.com/mbobakov/grpc-consul-resolver"
	"go.uber.org/zap"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"

	"cloudshop-api/user/global"
	"cloudshop-api/user/proto"
)

func InitServiceConn() {
	consul := global.ServerConfig.Consul
	userConn, err := grpc.Dial(
		fmt.Sprintf("consul://%s:%d/%s/?wait=14s", consul.Host, consul.Port, global.ServerConfig.Service.Name),
		grpc.WithTransportCredentials(insecure.NewCredentials()),
		grpc.WithDefaultServiceConfig(`{"loadBalancingPolicy": "round_robin"}`),
	)
	if err != nil {
		zap.S().Fatal("fail to connect Service")
	}
	userServiceClient := proto.NewUserClient(userConn)
	global.UserServiceClient = userServiceClient
}

The code can run and the grpc service and http service can be registered，also they can pass the health check:

But when the code is running, the terminal keeps showing the error:ERROR: [Consul resolver] Couldn't update client connection. error={bad resolver state}

As a result，I fail to call GRPC service，though my application can receive HTTP requests，which means Gin works well.