Giter Club home page Giter Club logo

fargo's Introduction

fargo

Netflix Eureka client in golang. Named for the show Eureka.

c = fargo.NewConn("http://127.0.0.1:8080/eureka/v2")
c.GetApps() // returns a map[String]fargo.Application

Testing

Tests can be executed using docker container. See the below section to build and start all the required containers. Once the Eureka containers are running, and fargo image is built then you can run the command as follows:

Run:

docker run --rm -v "$PWD":/go/src/github.com/hudl/fargo -w /go/src/github.com/hudl/fargo hudloss/fargo:master go test -v ./...

Note: If you are running bash for Windows add MSYS_NO_PATHCONV=1 at the beginning.

The tests may need to be run a couple of times to allow changes to propagate between the two Eureka servers. If the tests are failing, try running them again approximately 30 seconds later.

If you are adding new packages to godep you may want to update the hudloss/fargo image first.

Known Issues

Until this PR is in an official vagrant release, the Fedora 19 opscode box will fail when setting up private networks in vagrant 1.2.5 and later.

FAQ

Q: Is this a full client?

A: Not yet. It's very much a work in progress, and it's also being built with consideration for Go idioms, which means some Java-isms will never be included.

Q: Does it cache?

A: No, it does not support caching records.

Q: Can I integrate this into my Go app and have it manage hearbeats to Eureka?

A: Glad you asked, of course you can. Just grab an application (for this example, "TESTAPP")

// register a couple instances, and then set up to only heartbeat one of them
e, _ := fargo.NewConnFromConfigFile("/etc/fargo.gcfg")
app, _ := e.GetApp("TESTAPP")
// starts a goroutine that updates the application on poll interval
e.UpdateApp(&app)
for {
    for _, ins := range app.Instances {
        fmt.Printf("%s, ", ins.HostName)
    }
    fmt.Println(len(app.Instances))
    <-time.After(10 * time.Second)
}
// You'll see all the instances at first, and within a minute or two all the
// ones that aren't heartbeating will disappear from the list. Note that after
// calling `UpdateApp` there's no need to manually update

TODO

  • Actually do something with AWS availability zone info
  • Currently the load balancing is random, and does not give preference to servers within a zone.
  • Make releases available on gopkg.in

Hacking

Just Let Me Import Already

go get github.com/hudl/fargo

package main

import (
    "github.com/hudl/fargo"
)

func main() {
    e, _ := fargo.NewConnFromConfigFile("/etc/fargo.gcfg")
    e.AppWatchChannel
}

Adding Stuff

go test is your friend. I use the excellent goconvey library in addition to the standard lib's testing package. If you add something, write a test. If something is broken, write a test that reproduces your issue or post repro steps to the issues on this repository. The tests require that you have a eureka install and are designed to run against the included vagrant specs.

Verifying Releases

We're on semver and tag releases accordingly. The releases are signed and can be verified with git tag --verify vA.B.C.

Using Docker

Fargo is tested against two eureka versions, v1.1.147 and v1.3.1. To support testing, we provide Docker containers that supply Eureka locally. Here's how to get started.

  1. Clone Fargo
  2. If you don't have it, install Docker.
# Defaults to 1.1.147, can be set to 1.3.1
EUREKA_VERSION=1.1.147

cp docker-compose.override.yml.dist docker-compose.override.yml

docker-compose up -d

# Run the tests

Contributors

  • Ryan S. Brown (ryansb)
  • Carl Quinn (cquinn)

MIT License

The MIT License (MIT). Please see License File for more information.

fargo's People

Contributors

atharvai avatar camtendo avatar co-kevin avatar cquinn avatar damtur avatar eikenb avatar ryansb avatar sagikazarmark avatar seh avatar st3v avatar zeffron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fargo's Issues

Network requests sleep for 10 nanoseconds. Is that intentional?

Running staticcheck over fargo reveals one potential problem: in file rpc.go's netReq function, when a request fails with a temporary error, we sleep for ten nanoseconds. Note that the magnitude is specified without any unit, allowing its conversion to a time.Duration with the default unit of nanoseconds.

% staticcheck                                    
[...]/github.com/hudl/fargo/rpc.go:111:15: sleeping for 10 nanoseconds is probably a bug. Be explicit if it isn't: time.Sleep(10 * time.Nanosecond) (SA1004)

Error SA1004 has the following description:

Suspiciously small untyped constant in time.Sleep

What was the intended duration there? @ryansb added that statement in commit 4d053d5. Ryan, do you remember?

Preserve whether each of an instance's two ports is enabled

When Eureka either accepts an instance registration or offers an instance record back from a query by instance ID, application, or VIP address, the per-instance data structure exchanged describes two ports—the unadorned insecure port, and a secure port—both of which can be either enabled or disabled. The insecure port is enabled by default; the secure port is disabled by default.

At present, fargo can parse the "enabled" field of the composite "port" value in the JSON representation, but it discards the value immediately afterward (see fargo.Instance.UnmarshalJSON). In the XML representation, the "port" element has an "enabled" attribute that fargo ignores completely. The consequence is twofold:

  • instance consumers can't determine whether a given port is enabled, and
  • instance registrants can't specify whether a given port is enabled.

Not all Eureka clients pay attention to this information, but today I confirmed that Ribbon does. I've seen a few other clients that also check whether the secure port is enabled (apparently not caring whether the insecure port is disabled).

Given that fargo already defines the fargo.Instance.PortJ and fargo.Instance.SecurePortJ fields, I'd like to see if we can augment the marshaling procedures to preserve the port's enabled/disabled status round-trip through both the XML and JSON representations. Doing so would be mostly backward compatible, though registrants that were setting the fargo.Instance.PortJ.Enabled and fargo.Instance.SecurePortJ.Enabled field values to false or true respectively would find those values now being sent to the server—likely as originally intended.

EurekaConnection is missing the function in the latest tag 1.3.1

With go get in my project I am using the eureka connection to look up GetInstancesByVIPAddress.
Its missing in the latest tag pulled by dep though. Should I not be using GetInstancesByVIPAddress. Also Since I couldnt see the loadbalance implementation of returning a single load balanced instance when multiples are registered, I am currently using the shuffle with GetInstancesByVIPAddress.
Is that the correct approach?

Ease interpretation of errors that arise during registration operations

At present, the following functions return type error:

They can fail for many reasons, but a few stand out:

  • DeregisterInstance fails if the instance does not exist (HTTP status code 404).
  • DeregisterInstance fails if the instance did exist (HTTP status code 200).
    This is due to expecting status code 204, even though the Eureka documentation promises status code 200 for a successful deletion. This mismatch makes me wonder if anyone else is using this method.

To work around these problems, I had to write the following function to filter errors that arise when calling DeregisterInstance:

func deregistrationFailureIsInnocuous(err error) bool {
    msg := err.Error()
    return strings.Contains(msg, "200") || strings.Contains(msg, "404")
}

That is clear evidence of a gap in the interface.

Both ReregisterInstance and RegisterInstance can fail with status code 400 if Eureka deems the instance payload invalid. Note that it's not possible for registration to fail due to a collision with an existing instance with the same ID; Eureka replaces the existing registration.

It would be helpful for callers to be able to discern why these operations failed. Rather than introducing sentinel error values or types, I propose that we introduce a few exported functions:

  • InstanceWasMissing(error) bool
    True for failures in DeregisterInstance due to receiving status code 404
  • InstanceWasInvalid(error) bool
    True for failures in RegisterInstance and DeregisterInstance due to receiving status code 400

Do you have counter-proposals for these names? Are there other functions you can think of that would clarify what went wrong?

Cache: FAQ Update

Q: Does it cache?

A: Yes. There is a shared cache that is checked before calling out to Eureka servers.

I noticed in the FAQ there's references to a shared cache, but it appears to have been removed. This should be removed from the FAQ (or a cache added :)).

Thanks for fargo! It made a project significantly easier to write.

division by 0 panic

If you try to use DNSDiscovery without setting any ServiceUrls and the dns discovery fails, it causes a division by 0 panic, stemming from the choice([]string]string function in the connection go file. It depends on the passed in list not being empty without checking. A simple check in the choice() function would fix the panic.

UpdateApp assigning a struct from a goroutine is not safe

The UpdateApp method on EurekaConnection starts a goroutine that periodically unmarshals a new Application value, and then assigns it wholesale to an Application that represented the previous version of that Eureka application.

Per my note in file net.go, such assignment is not safe without either synchronization or use of atomic store and load operations. It's possible for a goroutine reading the Application value submitted to UpdateApp to observe a corrupted value. Per The Go Memory Model,

Reads and writes of values larger than a single machine word behave as multiple machine-word-sized operations in an unspecified order.

Since we can't orchestrate all the reads of the Application value submitted to UpdateApp, I suggest instead that UpdateApp offer a single-valued buffered channel from which interested clients can receive updates.

Accommodate data center info metadata for types other than "Amazon"

Fargo allows one to specify and retrieve Amazon-specific data center metadata, but does not accommodate any such metadata for other types of data centers. The Eureka XML schema for instance registration is loose here; it mentions that the Amazon-specific schema for the dataCenterInfo element's metadata element applies only when the name element is "Amazon".

metadata is only required if name is Amazon

That says that it's only required; what it really should say is that anything is tolerated, at the discretion of the data center-specific Java type that parses this metadata in the Eureka server. What actually happens is that the Eureka server parses the metadata element into a Map<String, String> and allows types implementing com.netflix.appinfo.DataCenterInfo to interpret that mapping as they see fit. It just so happens that the Eureka authors were only concerned with one concrete kind of data center. My company has implemented several more data center types in our Eureka fork.

I propose adding a field to fargo.DataCenterInfo—called "AlternateMetadata" of type map[string]string—that allows callers to send and receive this metadata for data center types other than "Amazon". Doing so is necessary for clients to decide on an instance ID that the server will accept. Without this concession, the server always uses the submitted host name. (See #39 for related discussion.)

I will submit a PR that takes care of the marshaling, but first I thought that we might like to discuss the need for and merits of this approach here first.

Support for zone stickiness

Making the Eureka selection sticky or at least stickier per zone would make some operations more reliable. Pick one at random and then keep it until there's a problem, or after a timeout period and then step through to the next one. This is more like what Netflix's Java client does.

Is this package still maintained?

The last commit in this repo is 5 years old, the last release is even older.

There are a couple outstanding issues and PRs (go modules, updating dependencies, etc) that would be nice to have to keep up with the world.

Do you accept PRs for those things? Or am I better off forking the library?

Thanks!

Questions about EurekaConnection.generateURL()

It seems like generateURL() could be use to completely assemble the URL from its slug parts just by passing the parts as varargs. But none of the calling code actually uses this feature, and instead assembles most of the URL with a couple different approaches.

E.g. in RegisterInstance(), two approaches in combination are used, Sprintf and later string cat:

slug := fmt.Sprintf("%s/%s", EurekaURLSlugs["Apps"], ins.App)
reqURL := e.generateURL(slug)
_, rcode, err := getBody(reqURL+"/"+ins.HostName, e.UseJson)

And it seems that using generateURL() to do the work would be simpler:

reqURL := e.generateURL(EurekaURLSlugs["Apps"], ins.App, ins.HostName)

Second question: what purpose does the EurekaURLSlugs[] lookup serve if it only ever maps a string to a nearly identical string. Maybe we can just remove it? Then the above would be:

reqURL := e.generateURL("apps", ins.App, ins.HostName)

I can test this and prepare a PR if it makes sense to you.

Support for AddMetadataString before instance registration

It seems like it would be cleaner for some use cases if the instance metadata could be setup first, and then just registered as payload in the instance registration. Adding additional metadata later would be fine as well, but in our case, we have a bunch of static values that we want to include with the instance for its whole lifespan.

I think the main task for this would be to track the instance registration state, and if not registered then skip the PUT part of AddMetadataString()

(I am also seeing some flakiness with the individual PUTs, and some metadata gets lost. Maybe Eureka propagating is sporadic, or setting individual values for an instance using different Eureka nodes is problematic. I need to debug this.)

Use vX.X.X for release tags

Please use the format vX.X.X for release tags. Go 1.11 modules will need the v prefix. Can you add new tags for the latest version with the prefix?

Supply the set of instances for a given VIP address

At present, fargo allows one to request and monitor a set of instances associated with a Eureka application, but there's no similar means to request or monitor of set of instances associated with a VIP address. Define new functions that use the /eureka/v2/vips/{address} and /eureka/v2/svips/{address} resources via the GET verb.

Sketching, we'd expand fargo's exported interface with the following:

  • Polling
    • func GetInstancesForVIPAddress(address string) ([]*Instance, error)
    • func GetInstancesForSecureVIPAddress(address string) ([]*Instance, error)
  • Consuming
    • type InstanceSetUpdate
    • func ScheduleInstanceSetUpdatesForVIPAddress(address string, await bool, done <-chan struct{}) <-chan InstanceSetUpdate
  • Caching
    • type InstanceSetSource
    • func NewInstanceSetSourceForVIPAddress(address string, await bool) *InstanceSetSource
    • func NewInstanceSetSourceForSecureVIPAddress(address string, await bool) *InstanceSetSource

The latter two groups mimic what we did in #37 for applications. The names get unwieldy; I could live with some contractions, such as "instances" instead of "instance set", and "SVIP" instead of "secure VIP address".

Best would be to just implement the first "polling" set, then move on with the other two.

Eurka set PollIntervalSeconds has Bug

at hargo latest version and golang at 1.10.3 , func UpdateApp() has a deadly bug,

func (e *EurekaConnection) UpdateApp(app *Application) {
go func() {
for {
log.Noticef("Updating app %s", app.Name)
err := e.readAppInto(app)
if err != nil {
log.Errorf("Failure updating %s in goroutine", app.Name)
}
<-time.After(time.Duration(e.PollInterval) * time.Second)
}
}()
}

func NewConnFromConfig(conf Config) (c EurekaConnection) {
c.ServiceUrls = conf.Eureka.ServiceUrls
c.ServicePort = conf.Eureka.ServerPort
if len(c.ServiceUrls) == 0 && len(conf.Eureka.ServerDNSName) > 0 {
c.ServiceUrls = []string{conf.Eureka.ServerDNSName}
}
c.Timeout = time.Duration(conf.Eureka.ConnectTimeoutSeconds) * time.Second
c.PollInterval = time.Duration(conf.Eureka.PollIntervalSeconds) * time.Second
c.PreferSameZone = conf.Eureka.PreferSameZone
if conf.Eureka.UseDNSForServiceUrls {
log.Warning("UseDNSForServiceUrls is an experimental option")
c.DNSDiscovery = true
c.DiscoveryZone = conf.Eureka.DNSDiscoveryZone
c.ServerURLBase = conf.Eureka.ServerURLBase
}
return c
}

The problem is <-time.After(time.Duration(e.PollInterval) * time.Second)

s := time.Duration(3) * time.Second
k := time.Duration(s) * time.Second
fmt.Println(s)
fmt.Println(k)

get
3s
833333h20m0s

so , they will be fixed to <-time.After(time.Duration(e.PollInterval))

About connect with feign

Description

I want to visit go api by feign, But i can't find the go server by feign.but go api has been register in the euraka

Register job at Eureka Server

Hello everyone,

I am trying to register my Golang client app in the Eureka java server, following this way:

func EurekaClienteRegister2() {
e := fargo.EurekaConnection{
ServiceUrls: []string{"http://10.0.2.110:9080/discovery-service/eureka"},
Timeout: 2,
}

i := fargo.Instance{
	HostName:         "http://10.0.2.248:8087/ged-service/",
	Port:             8087,
	App:              "ged-service-name",
	IPAddr:           "10.0.2.248",
	VipAddress:       "10.0.2.248",
	DataCenterInfo:   fargo.DataCenterInfo{Name: fargo.MyOwn},
	SecureVipAddress: "10.0.2.248",
	Status:           fargo.UP,
}

err := e.RegisterInstance(&i)
if err == nil {
	log.Println(err)
}

}

But, in the console Eureka server, it appear stranger:

image

And, the problem is that, my gateway don't find the client.
2021-11-09 11:39:16.246 WARN [gateway-service-name,de773706e662d245,de773706e662d245] 34176 --- [ctor-http-nio-3] o.s.c.l.core.RoundRobinLoadBalancer : No servers available for service: GED-SERVICE-NAME

Instance.InstanceId missing in 1.3.1 version when using with dep

When using fargo as go get github.com/hudl/fargo the Instance struct has a field InstanceId. However, this field is missing when declaring the dependency using dep. A few other fields are different as well. Below is the struct definition in each case.
Using go get:

// Instance [de]serializeable [to|from] Eureka [XML|JSON].
type Instance struct {
	InstanceId       string `xml:"instanceId" json:"instanceId"`  
	HostName         string `xml:"hostName" json:"hostName"`  
	App              string `xml:"app" json:"app"`  
	IPAddr           string `xml:"ipAddr" json:"ipAddr"`  
	VipAddress       string `xml:"vipAddress" json:"vipAddress"`  
	SecureVipAddress string `xml:"secureVipAddress" json:"secureVipAddress"`  
	Status           StatusType `xml:"status" json:"status"`  
	Overriddenstatus StatusType `xml:"overriddenstatus" json:"overriddenstatus"`  
	Port              int  `xml:"-" json:"-"`
	PortEnabled       bool `xml:"-" json:"-"`
	SecurePort        int  `xml:"-" json:"-"`
	SecurePortEnabled bool `xml:"-" json:"-"`
	HomePageUrl    string `xml:"homePageUrl" json:"homePageUrl"`
	StatusPageUrl  string `xml:"statusPageUrl" json:"statusPageUrl"`
	HealthCheckUrl string `xml:"healthCheckUrl" json:"healthCheckUrl"`
	CountryId      int64          `xml:"countryId" json:"countryId"`
	DataCenterInfo DataCenterInfo `xml:"dataCenterInfo" json:"dataCenterInfo"`
	LeaseInfo LeaseInfo        `xml:"leaseInfo" json:"leaseInfo"`
	Metadata  InstanceMetadata `xml:"metadata" json:"metadata"`
	UniqueID func(i Instance) string `xml:"-" json:"-"`
}

Using dep

// Instance [de]serializeable [to|from] Eureka XML.
type Instance struct {
	XMLName          struct{} `xml:"instance" json:"-"`
	HostName         string   `xml:"hostName" json:"hostName"`
	App              string   `xml:"app" json:"app"`
	IPAddr           string   `xml:"ipAddr" json:"ipAddr"`
	VipAddress       string   `xml:"vipAddress" json:"vipAddress"`
	SecureVipAddress string   `xml:"secureVipAddress" json:"secureVipAddress"`
	Status           StatusType `xml:"status" json:"status"`
	Overriddenstatus StatusType `xml:"overriddenstatus" json:"overriddenstatus"`
	Port        int  `xml:"port" json:"-"`
	PortJ       Port `json:"port" xml:"-"`
	SecurePort  int  `xml:"securePort" json:"-"`
	SecurePortJ Port `json:"securePort" xml:"-"`
	HomePageUrl    string `xml:"homePageUrl" json:"homePageUrl"`
	StatusPageUrl  string `xml:"statusPageUrl" json:"statusPageUrl"`
	HealthCheckUrl string `xml:"healthCheckUrl" json:"healthCheckUrl"`
	CountryId      int64          `xml:"countryId" json:"countryId"`
	DataCenterInfo DataCenterInfo `xml:"dataCenterInfo" json:"dataCenterInfo"`
	LeaseInfo LeaseInfo        `xml:"leaseInfo" json:"leaseInfo"`
	Metadata  InstanceMetadata `xml:"metadata" json:"metadata"`
	UniqueID func(i Instance) string `xml:"-" json:"-"`
}

The Gopkg.toml snippet is

.....
[[constraint]]
  name = "github.com/hudl/fargo"
  version = "1.3.1"
.....

Which of these is the correct one and should be used?

Missing InstanceId into fargo.Instance struct

Hello,
Is it possible to add into fargo.Instance this field:

InstanceId       string   `xml:"instanceId" json:"instanceId"`

because if I use UniqueID func(i Instance) string xml:"-" json:"-"

I have this error into eureka console:

Not Found (Renew): xxxx - 10.193.204.167:xxxx:9001:2016-07-08 18:30:19.779

Thx in adv

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.