Giter Club home page Giter Club logo

ttn-gateway-connector's Introduction

The Things Network Gateway Connector

Embedded C library for The Things Gateway and Linux-based gateways to connect with The Things Network Router.

Introduction

The connector is an embedded C library intended to use on The Things Gateway and Linux-based gateways to connect with The Things Network.

The connector uses a MQTT connection using the Paho Embedded C/C++ Library. It uses MQTTPacket and MQTTClient-C which are not reliant on particular libraries for networking, threading or memory management.

Messages are packed using Protocol Buffers, a language-neutral, platform-neutral extensible mechanism for serializing structured data. The protos used are The Things Network API compiled to C sources and headers.

Is This Necessary?

Yes. This library provides the following advantages over Semtech's packet forwarder reference implementation:

  1. When using TCP/IP, there is an open and more reliable connection with The Things Network as compared to UDP which suffers from congestion and firewalls. It is my intention to use port 80 instead of 1883 (default MQTT) to minimize outgoing firewall restrictions
  2. When using MQTT, we build on top of a proven and well-supported messaging protocol as compared to custom headers and breaking changes on message protocol level
  3. When using authentication, we can identify gateways according to their ID and security key as compared to user-defined EUIs which deemed unreliable
  4. When using Protocol Buffers, we use proven and well-supported (un)packing and we are binary compatible with The Things Network back-end while minimizing payload and allow for protocol changes

Building

The connector requires a C compiler and protobuf-c to be installed.

Clone the source of the forked Paho Embedded C/C++ Library. Set its relative path to PAHO_SRC in config.mk (copy from config.mk.in). Build the library with make and install with sudo make install.

Build ttn-gateway-connector:

make

Example

#include <stdio.h>
#include <signal.h>
#include <stdint.h>
#include <unistd.h>
#include <stdlib.h>
#include <time.h>
#include "connector.h"

int running = 1;

void stop(int sig)
{
   signal(SIGINT, NULL);
   running = 0;
}

void print_downlink(Router__DownlinkMessage *msg, void *arg)
{
   if (msg->has_payload)
   {
      printf("down: have %zu bytes downlink\n", msg->payload.len);
      switch (msg->protocol_configuration->protocol_case)
      {
      case PROTOCOL__TX_CONFIGURATION__PROTOCOL_LORAWAN:
      {
         Lorawan__TxConfiguration *lora = msg->protocol_configuration->lorawan;
         printf("down: modulation: %d, data rate: %s, bit rate: %d, coding rate: %s, fcnt: %d\n",
                lora->modulation, lora->data_rate, lora->bit_rate, lora->coding_rate, lora->f_cnt);

         Gateway__TxConfiguration *gtw = msg->gateway_configuration;
         printf("down: timestamp: %d, rf chain: %d, frequency: %llu, power: %d, polarization inversion: %d, frequency deviation: %d\n",
                gtw->timestamp, gtw->rf_chain, gtw->frequency, gtw->power, gtw->polarization_inversion, gtw->frequency_deviation);

         break;
      }
      default:
         printf("down: invalid protocol %d\n", msg->protocol_configuration->protocol_case);
         break;
      }
   }
}

int main(int argc, char **argv)
{
   srand(time(NULL));

   signal(SIGINT, stop);
   signal(SIGTERM, stop);

   // Initialize the TTN gateway with ID test
   TTN *ttn;
   ttngwc_init(&ttn, "test", &print_downlink, NULL);
   if (!ttn)
   {
      printf("failed to initialize TTN gateway\n");
      return -1;
   }

   // Connect to the test broker. The secret key is NULL for simplicity
   printf("connecting...\n");
   int err = ttngwc_connect(ttn, "localhost", 1883, NULL);
   if (err != 0)
   {
      printf("connect failed: %d\n", err);
      ttngwc_cleanup(ttn);
      return err;
   }
   printf("connected\n");

   // Start the loop
   int i = 0;
   while (running)
   {
      // Send gateway status. The only parameter we're sending is the time
      Gateway__Status status = GATEWAY__STATUS__INIT;
      status.has_time = 1;
      status.time = ++i;
      err = ttngwc_send_status(ttn, &status);
      if (err)
         printf("status: send failed: %d\n", err);
      else
         printf("status: sent with time %d\n", i);

      // Enter the payload
      unsigned char buf[] = {0x1, 0x2, 0x3, 0x4, 0x5};
      Router__UplinkMessage up = ROUTER__UPLINK_MESSAGE__INIT;
      up.has_payload = 1;
      up.payload.len = sizeof(buf);
      up.payload.data = buf;

      // Set protocol metadata
      Protocol__RxMetadata protocol = PROTOCOL__RX_METADATA__INIT;
      protocol.protocol_case = PROTOCOL__RX_METADATA__PROTOCOL_LORAWAN;
      Lorawan__Metadata lorawan = LORAWAN__METADATA__INIT;
      lorawan.has_modulation = 1;
      lorawan.modulation = LORAWAN__MODULATION__LORA;
      lorawan.data_rate = "SF9BW250";
      lorawan.coding_rate = "4/5";
      lorawan.has_f_cnt = 1;
      lorawan.f_cnt = i;
      protocol.lorawan = &lorawan;
      up.protocol_metadata = &protocol;

      // Set gateway metadata
      Gateway__RxMetadata gateway = GATEWAY__RX_METADATA__INIT;
      gateway.has_timestamp = 1;
      gateway.timestamp = 10000 + i * 100;
      gateway.has_rf_chain = 1;
      gateway.rf_chain = 5;
      gateway.has_frequency = 1;
      gateway.frequency = 86800;
      up.gateway_metadata = &gateway;

      // Send uplink message
      err = ttngwc_send_uplink(ttn, &up);
      if (err)
         printf("up: send failed: %d\n", err);
      else
         printf("up: sent with timestamp %d\n", i);

      sleep(rand() % 10);
   }

   printf("disconnecting\n");
   ttngwc_disconnect(ttn);
   ttngwc_cleanup(ttn);

   return 0;
}

Testing

There is an example Router in examples/router which is written in Go. This requires the Go compiler, see here:

cd examples/router
go run main.go

The sample application src/test.c publishes a message to the MQTT broker on localhost every second as gateway test.

make test

Subscribe to the topic, for example: mosquitto_sub -t test/+ -d. The output should look like this:

➜  ~ mosquitto_sub -t 'test/+' -d
Client mosqsub/14673-Johans-Ma sending CONNECT
Client mosqsub/14673-Johans-Ma received CONNACK
Client mosqsub/14673-Johans-Ma sending SUBSCRIBE (Mid: 1, Topic: test/+, QoS: 0)
Client mosqsub/14673-Johans-Ma received SUBACK
Subscribed (mid: 1): 0
Client mosqsub/14673-Johans-Ma received PUBLISH (d0, q0, r0, m0, 'test/status', ... (2 bytes))

Client mosqsub/14673-Johans-Ma received PUBLISH (d0, q0, r0, m0, 'test/up', ... (43 bytes))

Z
XSF9BW250r4/5xb
               X�R����
Client mosqsub/14673-Johans-Ma received PUBLISH (d0, q0, r0, m0, 'test/status', ... (2 bytes))

Client mosqsub/14673-Johans-Ma received PUBLISH (d0, q0, r0, m0, 'test/up', ... (43 bytes))

Z
XSF9BW250r4/5xb
               X�S����

Next Steps

  • Implement platform specific Timer, Mutex, Condition and Network for Microchip Harmony

Update

Optional: Update Protos

To update the protos to the latest version of The Things Network API, in your $GOPATH/src or sources folder:

mkdir -p github.com/gogo
cd github.com/gogo
git clone https://github.com/gogo/protobuf.git
cd ../..
mkdir -p github.com/TheThingsNetwork
cd github.com/TheThingsNetwork
git clone https://github.com/TheThingsNetwork/ttn.git
cd ttn

If you do not have Go installed or if you do not have cloned the repository to your $GOPATH, set the GOPATH variable in config.mk to the absolute or relative path where you executed the commands above.

make proto

License

Source code for The Things Network is released under the MIT License, which can be found in the LICENSE file. A list of authors can be found in the AUTHORS file.

ttn-gateway-connector's People

Contributors

htdvisser avatar johanstokking avatar kersing avatar martinichka avatar sunfmin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ttn-gateway-connector's Issues

Error configuring protobuf-c

Hi,

Since there is not that good information of configuring, compiling and installing protobuf and protobuf-c, can you please provide aditional information?

Here is my error message when configuring protobuf-c:

checking google/protobuf/compiler/command_line_interface.h usability... no
checking google/protobuf/compiler/command_line_interface.h presence... no
checking for google/protobuf/compiler/command_line_interface.h... no
ERROR: Installing protobuf-c library. Please check env_protobuf.log

Thanks,

Cheers,
Martin

Error during make of gateway-connector

Make spits out a couple of errors, how to handle? I've tried both https://github.com/TheThingsNetwork/paho.mqtt.embedded-c and https://github.com/eclipse/paho.mqtt.embedded-c

root@gateway:/home/ubuntu/ttn-gateway-connector# make
mkdir -p obj/
cc -fPIC -Wall -g -O2 -Isrc -I../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src -I../paho.mqtt.embedded-c-1.0.0/MQTTPacket/src -Isrc/github.com/gogo/protobuf/protobuf -Isrc/github.com/TheThingsNetwork -DMQTT_TASK -I../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/linux -c src/connector.c -o obj/connector.o
In file included from src/network.h:7:0,
from src/connector.c:5:
../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/MQTTClient.h:22:10: error: empty filename in #include
#include "" //Platform specific implementation header file
^
../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/MQTTClient.h:32:15: error: unknown type name ‘Timer’
void NewTimer(Timer*);
^
../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/MQTTClient.h:67:26: error: unknown type name ‘Network’
void MQTTClient(Client*, Network*, unsigned int, unsigned char*, size_t, unsigned char*, size_t);
^
../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/MQTTClient.h:87:5: error: unknown type name ‘Network’
Network* ipstack;
^
../paho.mqtt.embedded-c-1.0.0/MQTTClient-C/src/MQTTClient.h:88:5: error: unknown type name ‘Timer’
Timer ping_timer;
^
In file included from src/network.h:11:0,
from src/connector.c:5:
src/session.h:10:3: error: unknown type name ‘Network’
Network network;
^
src/session.h:11:3: error: unknown type name ‘MQTTClient’
MQTTClient client;

...

int MQTTSubscribe (Client*, const char*, enum QoS, messageHandler);
^
src/connector.c:113:9: error: too many arguments to function ‘MQTTSubscribe’
err = MQTTSubscribe(&session->client, session->downlink_topic, QOS_DOWN,
^
In file included from src/network.h:7:0,
from src/connector.c:5:

Proto file absence

Hi,

I am trying to implement this on an embedded system, not embedded linux and I want to do it with nano-pb but I would need the actual proto files to test.

Are they available?

Regards,
Kiogora.

Link broken

In README.md the following text is used, the link is broken

The protos used are [The Things Network API](https://github.com/TheThingsNetwork/ttn/tree/refactor/api) compiled to C sources and headers.

Instructions missing paho.mqtt.golang

Trying to get the router example to run, as far as I can find paho.mqtt.golang needs to be installed using go get, this is not mentioned in the README.me

go run main.go
main.go:15:2: cannot find package "github.com/eclipse/paho.mqtt.golang" in any of:
        /usr/lib64/go/src/github.com/eclipse/paho.mqtt.golang (from $GOROOT)
        /home/johnny/ACE/go/src/github.com/eclipse/paho.mqtt.golang (from $GOPATH)
go get github.com/eclipse/paho.mqtt.golang
go run main.go
panic: Network Error : dial tcp [::1]:1883: getsockopt: connection refused

Where the latest error is probably related to the lack of a mqtt server

Add legal stuff

  • Copyright headers
  • LICENSE file
  • License section in README.md
  • AUTHORS file

Missing: src/github.com/TheThingsNetwork/ttn/api/trace/*

$ make
mkdir -p obj/
cc -fPIC -Wall -g -O2 -Isrc -I../paho.mqtt.embedded-c/MQTTClient-C/src -I../paho.mqtt.embedded-c/MQTTPacket/src -Isrc/github.com/gogo/protobuf/protobuf -Isrc/github.com/TheThingsNetwork -DMQTT_TASK -I../paho.mqtt.embedded-c/MQTTClient-C/src/linux -c src/connector.c -o obj/connector.o
In file included from src/connector.h:16:0,
from src/network.h:10,
from src/connector.c:5:
src/github.com/TheThingsNetwork/ttn/api/router/router.pb-c.h:22:38: fatal error: ttn/api/trace/trace.pb-c.h: No such file or directory
compilation terminated.
Makefile:39: recipe for target 'obj/connector.o' failed
make: *** [obj/connector.o] Error 1

Error compiling the test sample

I'm trying to compile test.c and get some errors. What am I doing wrong?

root@gateway:/home/ubuntu/ttn-gateway-connector# make test
LD_LIBRARY_PATH=bin:/usr/local/lib/ ./bin/ttn-gateway-connector_test
connecting...
connect failed: -1
Makefile:57: recipe for target 'test' failed
make: *** [test] Error 255
root@gateway:/home/ubuntu/ttn-gateway-connector# cd src
root@gateway:/home/ubuntu/ttn-gateway-connector/src# make test
cc test.c -o test
In file included from connector.h:15:0,
from test.c:10:
github.com/TheThingsNetwork/ttn/api/gateway/gateway.pb-c.h:17:58: fatal error: github.com/gogo/protobuf/gogoproto/gogo.pb-c.h: No such file or directory
compilation terminated.
: recipe for target 'test' failed
make: *** [test] Error 1
root@gateway:/home/ubuntu/ttn-gateway-connector/src# cd ..
root@gateway:/home/ubuntu/ttn-gateway-connector# make ./src/test
cc -fPIC -Wall -g -O2 -Isrc -I../paho.mqtt.embedded-c/MQTTClient-C/src -I../paho.mqtt.embedded-c/MQTTPacket/src -Isrc/github.com/gogo/protobuf/protobuf -Isrc/github.com/TheThingsNetwork -DMQTT_TASK -I../paho.mqtt.embedded-c/MQTTClient-C/src/linux src/test.c -o src/test
/tmp/ccJy4YED.o: In function main': /home/ubuntu/ttn-gateway-connector/src/test.c:54: undefined reference to ttngwc_init'
/home/ubuntu/ttn-gateway-connector/src/test.c:62: undefined reference to ttngwc_connect' /home/ubuntu/ttn-gateway-connector/src/test.c:114: undefined reference to ttngwc_send_uplink'
/home/ubuntu/ttn-gateway-connector/src/test.c:77: undefined reference to ttngwc_send_status' /home/ubuntu/ttn-gateway-connector/src/test.c:124: undefined reference to ttngwc_disconnect'
/home/ubuntu/ttn-gateway-connector/src/test.c:125: undefined reference to ttngwc_cleanup' /home/ubuntu/ttn-gateway-connector/src/test.c:65: undefined reference to ttngwc_cleanup'
/home/ubuntu/ttn-gateway-connector/src/test.c:128: undefined reference to gateway__status__descriptor' /home/ubuntu/ttn-gateway-connector/src/test.c:128: undefined reference to router__uplink_message__descriptor'
/home/ubuntu/ttn-gateway-connector/src/test.c:128: undefined reference to protocol__rx_metadata__descriptor' /home/ubuntu/ttn-gateway-connector/src/test.c:128: undefined reference to lorawan__metadata__descriptor'
/home/ubuntu/ttn-gateway-connector/src/test.c:128: undefined reference to `gateway__rx_metadata__descriptor'
collect2: error: ld returned 1 exit status
: recipe for target 'src/test' failed
make: *** [src/test] Error 1
root@gateway:/home/ubuntu/ttn-gateway-connector#

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.