thethingsarchive / gateway-connector-bridge Goto Github PK

View Code? Open in Web Editor NEW

68.0 5.0 27.0 540 KB

Bridge between Gateway Connector (protobufs over MQTT) and gRPC

License: MIT License

Go 89.39% Makefile 0.82% JavaScript 9.18% HTML 0.25% CSS 0.27% Dockerfile 0.10%

mqtt amqp grpc bridge iot internet-of-things lorawan lora lora-server lorawan-server

gateway-connector-bridge's People

Contributors

Stargazers

Watchers

gateway-connector-bridge's Issues

Panic in UplinkMessage MarshalTo

Version: Docker - thethingsnetwork/gateway-connector-bridge:v2.3.1
Redis ON
MQTT ON
UDP ON
Status ON
Debug OFF

panic: runtime error: index out of range

goroutine 114 [running]:
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/trace.(*Trace).MarshalTo(0xc4201ea5b0, 0xc42048aaf8, 0x3e, 0x3e, 0x6e, 0x57, 0x3)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/trace/trace.pb.go:261 +0x936
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/trace.(*Trace).MarshalTo(0xc4201ea7e0, 0xc42048aaa1, 0x95, 0x95, 0xc5, 0xa1, 0x0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/trace/trace.pb.go:294 +0x849
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router.(*UplinkMessage).MarshalTo(0xc420194e70, 0xc42048aa00, 0x136, 0x136, 0x136, 0x136, 0xcb34e0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/router.pb.go:1470 +0x514
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router.(*UplinkMessage).Marshal(0xc420194e70, 0x7f04bd9791e8, 0xc420194e70, 0x7f04bd979220, 0xc420194e70, 0x1)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/router.pb.go:1420 +0x7f
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/golang/protobuf/proto.(*Buffer).Marshal(0xc420298008, 0x7f04bd9791e8, 0xc420194e70, 0xc420479000, 0x0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/golang/protobuf/proto/encode.go:264 +0x7a
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc.protoCodec.marshal(0xcb34e0, 0xc420194e70, 0xc420298000, 0x14eed80, 0xef88, 0x114, 0x119, 0x120)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc/codec.go:63 +0xda
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc.protoCodec.Marshal(0xcb34e0, 0xc420194e70, 0x443747, 0xc420569b10, 0x42ab88, 0xc4204eee94, 0xc42052ea80)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc/codec.go:73 +0x6e
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc.(*protoCodec).Marshal(0x12c79c8, 0xcb34e0, 0xc420194e70, 0xc42031e300, 0xc4206147c0, 0x43e5eb, 0xc420021900, 0x3)
	<autogenerated>:1 +0x48
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc.encode(0x1250500, 0x12c79c8, 0xcb34e0, 0xc420194e70, 0x0, 0x0, 0x0, 0x0, 0xc4201f5b60, 0xc420569ee0, ...)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc/rpc_util.go:302 +0x28e
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc.(*clientStream).SendMsg(0xc42013e580, 0xcb34e0, 0xc420194e70, 0x0, 0x0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/google.golang.org/grpc/stream.go:370 +0x111
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/go-utils/grpc/restartstream.(*restartingStream).SendMsg(0xc420392a80, 0xcb34e0, 0xc420194e70, 0xc4202ac0f0, 0xc420129ce0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/go-utils/grpc/restartstream/restart.go:125 +0x2f6
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router.(*routerUplinkClient).Send(0xc420436680, 0xc420194e70, 0xc420569e38, 0x0)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/router.pb.go:1102 +0x49
github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/routerclient.(*Client).NewGatewayStreams.func2(0xc4206aaa20, 0xc4206aaae0, 0xc4206aab40, 0xc42044e0e0, 0x1257de0, 0xc4206082c8, 0xc42001ee01, 0xc420478420, 0xc420478400, 0xc4206ac690)
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/routerclient/client.go:316 +0x61e
created by github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/routerclient.(*Client).NewGatewayStreams
	/go/src/github.com/TheThingsNetwork/gateway-connector-bridge/vendor/github.com/TheThingsNetwork/api/router/routerclient/client.go:200 +0x439

Gateways are "reconnected" before MQTT connection is ready

HTTP 404 in Dokumentation

Under protocol router.UplinkMessage, router.DownlinkMessage and gateway.Status the documentation refers to Pages give a 404.

Link targets:
https://github.com/TheThingsNetwork/ttn/blob/develop/api/router/router.proto
https://github.com/TheThingsNetwork/ttn/blob/develop/api/gateway/gateway.proto

Flaky MQTT/AMQP tests on Travis

Hide MQTT password in log

When setting a MQTT password it is logged. We should no log passwords.

I can create a PR for this.

ctx.WithField("Username", parts[1]).WithField("Password", parts[2]).WithField("Address", parts[3]).Infof("Initializing MQTT")

Add DEBUG flag for setting log level

Log errors with wrong configured Router

The config parameter for the TTN Router must contain a slash as in: BRIDGE_TTN_ROUTER: discovery.thethingsnetwork.org:1900/the-router-id

If you miss the slash it's silently ignored.

See:

for _, ttnRouter := range ttnRouters {
		if ttnRouter == "disable" {
			continue
		}
		parts := strings.Split(ttnRouter, "/")
		if len(parts) == 2 {
			ctx.WithField("DiscoveryServer", parts[0]).WithField("RouterID", parts[1]).Infof("Initializing TTN Router")
			router, err := ttn.New(ttn.RouterConfig{
				DiscoveryServer: parts[0],
				RouterID:        parts[1],
			}, ctx, func(gatewayID string) string {
				token, err := authBackend.GetToken(gatewayID)
				if err != nil && err != auth.ErrGatewayNotFound {
					ctx.WithField("GatewayID", gatewayID).WithError(err).Debug("Could not get token for Gateway")
					return ""
				}
				return token
			})
			if err != nil {
				ctx.WithError(err).Warnf("Could not initialize TTN router %s", ttnRouter)
				continue
			}
			bridge.AddNorthbound(router)
		}
	}

Please add a warning log :)

Use buffered channel in UDP backend

Write to a buffered channel instead of spawning a goroutine which may block on channel write. This avoids memory leaks and out of memory panics when the channels cannot be flushed. For concurrency, start multiple goroutines, but not one per message.

https://github.com/TheThingsNetwork/gateway-connector-bridge/blob/master/backend/pktfwd/backend.go#L238

Add Default for BRIDGE_UDP or warn if missing

There should be some log about wrongly/not configured BRIDGE_UDP. Currently it's silently ignored and the bridge is just not working without helpful logs.

Use authentication in inject middleware

Add authentication to disconnect message

In order to prevent any gateway from disconnecting any other gateway, we should put authentication in both the connect and disconnect messages.

Bridge runs at 100% CPU

Usually the issue resolves itself or the bridge gets killed. Will get a pprof next time it happens.

Insert location source in metadata

See TheThingsArchive/ttn#635

Write Documentation

Add statistics

Write DummyHTTPServer Tests

Refresh token faster

Currently, when an attribute of the gateway changes, it takes too long for that change to propagate to the community pages.

This is due to the way the connector bridge caches the gateway information.
If I'm not mistaken, it fetches it once (via the gateway token) and then only refreshes it when the token expires. The current expiry is set to 3 months.

Changing an owner, the location, etc. will therefore only update after 3 months.

This is very confusing for users and the source of most support requests we get.
Can we shorten this?

Add packet_forwarder as backend

Update gRPC vendor

It's now v1.6.0, which is pretty old and 9 minor releases behind.

Expose different UDP endpoints for different regions

Move connect and disconnect to gateway topic and set retain=1

On Gateway connect:

publish ConnectMessage on <gateway-id>/conn topic (retained=1)
set will to DisconnectMessage on <gateway-id>/conn topic (retained=1)

This way, the bridge can see whether the gateway is connected or disconnected when it restarts.

Unfortunately, it's not possible to detect wheter a protobuf is one or the other, so we will need a ConnectDisconnect message with a oneof.

Keep track of gateway connection state between bridge restarts

Select RSIG instead of RSSI and SNR when present

When the RSIG is present but not RSSI the RSSI value transmitted to the network is 0. This cause problem with Location services, device monitoring and possibly ADR.

Would need to check for RSIG and use it instead of RSSI/SNR.

thethingsarchive / gateway-connector-bridge Goto Github PK

gateway-connector-bridge's People

Contributors

Stargazers

Watchers

Forkers

gateway-connector-bridge's Issues

Recommend Projects

Recommend Topics

Recommend Org