fitbit / golden-gate Goto Github PK

First of all, I would like to thank you all for making this project publicly available. It's amazing to see these kind of projects being open sourced as, in my humble opinion, could be differentiator elements with other competitors in the same market. Even so, you decided to open source it and I firmly believe it's a much better strategy than keeping it behind your fences!

So just a big shoutout to the people it make this a reality 🙏!

Recently we found ourselves in the same situation as yours, trying to see how would be the best way "democratize" the communication with the IoT park we already have deployed. As the kind of IoT devices are getting more and more complex, the need for a better and customisable/composable RPC system is becoming a must.

Before landing to this project, we reach to the same destination as you did, that is:

• Don't reinvent the wheel and use standard protocols: CoAP, DTLS
• Custom BLE layer to achieve reliability with Android devices
• Ability to communicate between different parties ({Desktop,Web,Android,iOS,Cloud}<->IoT)
• Have a clear separation of concerns to make layers pluggable (i.e. remove DTLS because we know the transport/application already provides end-to-end security)
• ...

We are in the early stage of this evaluation but we've big concerns about where does this project laids on regarding to Project Connected Home over IP (CHIP) as it seems to be the final standarisation the IoT world needed for years.

I'm not personally part of the alliance but it seems the idea is to embrace Thread+Zigbee (solving most requirements for all layers link/network<->application) and make it as open as possible through CHIP. That probably means that they are also going to somehow make the "ZigBee Cluster Library" theirs and adapt it to solve most of the application use cases. Although this doesn't solve all of our use cases I wanted to know what do you think about this, as you probably have a clear long term vision on this.

As an outsider I'm (I didn't found the time to get deeper in the code/docs), it seems like you decided to use standard protocols and create a custom solution around them. In case CHIP (& Thread) finally wons this IoT protocol battle, where does this project stands?

Thanks again and keep up the good work! I will follow it closely 👀!

https://github.com/azure/azure-sdk-for-js/blob/main/common/scripts/install-run-rush.js

Hello! GG has a very promising and unique design that I'd like to incorporate into future projects. The docs mention Zephyr and I found references to a WebAssembly target in the src. Can you share timing or a roadmap for these two ports?

Thanks!

Apologies if format is incorrect, found no documentation regarding desired format for issues.

Was looking to build into Zephyr RTOS, most likely on a nRF52 or nRF53, documentation file exists but seems to be empty. Is there a current or planned port at some future time?

Hi! We are looking for an alternative to our current IoT communication stack and we just came across this project. I've been playing around with it for quite a while and it looks amazing.

I would like to propose this framework to our firmware department but I have some doubts regarding code size. According to your post in https://eng.fitbit.com/introducing-project-golden-gate/ the framework is prepared to work on "a few kilobytes of RAM and flash and low power MCUs" but I've been playing around with the provided examples and I feel like the size of the binaries I'm getting is a bit high.

"Gattlink blast over udp" binary for example takes 182 KB and even though I managed to get it down to 125 KB by modifying the provided linux profile (disabled some unused features like DTLS or Lwip), it's still heavier than our current stack (which does include DTLS). I'm not experienced in C, CMake or IoT devices so it can probably be reduced a lot more. Thats why I would like to know the sizes you are working with, what is the size I should expect for a full featured (CoAP, DTLS, Gattlink and Sockets) GG communication stack?. We are trying not to go past ~100 KB and knowing the binary sizes you are working with will be helpful.

Thanks a lot.

Hello there!

I am trying to work through some of the examples building and running this application. Specifically, I am trying to run Stack Tool - gg-stack-tool - with Bluetooth on my MacOS (Big Sur) with

gg-stack-tool --bottom bluetooth connect hub DSNG

However, I keep getting a failure that I was able to track back using lldb to MacOS permissioning issues:

2021-01-20 12:27:15.983030-0800 gg-stack-tool[82249:687439] [access] This app has crashed because it attempted to access privacy-sensitive data without a usage description. The app's Info.plist must contain an NSBluetoothAlwaysUsageDescription key with a string value explaining to the user how the app uses this data.

I have attempted updating every instance of an Info.plist in the repository with this key

find . -name Info.plist | xargs -IPLIST plutil -replace NSBluetoothAlwaysUsageDescription -string 'Please allow Bluetooth so this application can work!' PLIST

However, even after rebuilding the entire application - inv native.clean followed by inv native.build - I am still plagued by the same error.

Where can I add this permission to have the build system pick it up for the Stack Tool?

Thanks in advance for your help. Appreciate your time!

Hi development team,

I'm unsure if my issue is related to the missing support of Windows. Nevertheless, I'd like to notify you about it.

After downloading the project and importing it to Android Studio for APK compilation, I received the following error about a missing library:

Task :GoldenGateBindings:externalNativeBuildDebug FAILED
Build xp_x86
ninja: Entering directory `C:\tmp\gg\golden-gate\platform\android\goldengate\GoldenGateBindings.cxx\cmake\debug\x86'
ninja: error: '../../../../../../../../xp/build/cmake/android/x86/libgg.so', needed by '../../../../build/intermediates/cmake/debug/obj/x86/libxp.so', missing and no known rule to make it

Any idea on how to fix that?

Hi there!

We are getting an exception when using the example app for Android and Gatt/UDP/DTLS and CoAP. I was wondering if you have an hint for us on how to fix that. Is that some sort of device issue with missing values?

BitGattPeer$setupNotifications-[main] fail to subscribe notification...
java.lang.IllegalStateException: descriptor must not be null
	at com.fitbit.bluetooth.fbgatt.rx.client.BitGattPeer$writeDescriptorCharacteristic$1.call(BitGattPeer.kt:153)
	at com.fitbit.bluetooth.fbgatt.rx.client.BitGattPeer$writeDescriptorCharacteristic$1.call(BitGattPeer.kt:32)
	at io.reactivex.internal.operators.single.SingleDefer.subscribeActual(SingleDefer.java:36)
	at io.reactivex.Single.subscribe(Single.java:3603)
	at io.reactivex.internal.operators.single.SingleFlatMap$SingleFlatMapCallback.onSuccess(SingleFlatMap.java:84)
	at io.reactivex.internal.operators.single.SingleMap$MapSingleObserver.onSuccess(SingleMap.java:64)
	at io.reactivex.internal.operators.single.SingleCreate$Emitter.onSuccess(SingleCreate.java:67)
	at com.fitbit.bluetooth.fbgatt.rx.GattTransactionExtKt$runTxReactive$1$1.onTransactionComplete(GattTransactionExt.kt:24)
	at com.fitbit.bluetooth.fbgatt.GattTransaction$1.onTransactionComplete(GattTransaction.java:373)
	at com.fitbit.bluetooth.fbgatt.GattTransaction.callCallbackWithTransactionResultAndRelease(GattTransaction.java:334)
	at com.fitbit.bluetooth.fbgatt.tx.SubscribeToCharacteristicNotificationsTransaction.lambda$transaction$0$SubscribeToCharacteristicNotificationsTransaction(SubscribeToCharacteristicNotificationsTransaction.java:100)
	at com.fitbit.bluetooth.fbgatt.tx.-$$Lambda$SubscribeToCharacteristicNotificationsTransaction$XxolJDFBYeyzUBs_rl-S9nO1zxw.run(Unknown Source:6)
	at android.os.Handler.handleCallback(Handler.java:883)
	at android.os.Handler.dispatchMessage(Handler.java:100)
	at android.os.Looper.loop(Looper.java:214)
	at android.app.ActivityThread.main(ActivityThread.java:7386)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:980)
-java.lang.IllegalStateException: descriptor must not be null

Hey Maintainers!

Generally inquiring if there is a timeline/roadmap to support the HTTP/TLS/TCP components of the Golden Gate stack that are referenced in the documentation but not yet implemented? Furthermore, if you can comment, is this something that is slated to be externalized or would this project benefit from net-new contributions to those parts of the stack? Please advise.

Thanks!

Hi,

We are trying to migrate golden-gate to a Cotex-M4 chip (GR5515, this chip is very cheap). I notice there are source code for NRF52840 in golden-gate/platform/mynewt/apps/gg-tool/apps/gg_central/src/ . But I did't see any code from NRF52840 lib in main.c. How does golden-gate migrate to NRF52840 board? Could you explain the blueprint behind the code?

Thanks!

Hello developers,

after compiling the demo app for Android and using Gattlink+UDP+DTLS (we will require the same setup and encryption or even something with certificates), I noticed that the throughput is very slow with about 1950 B/s only.

By default, increasing the packet size would allow me to reduce protocol overhead and gain more speed/throughput. I was wondering, which values you would recommend me for tweaking here?

In the "Link Controller Setup" I noticed that the MTU (for BT) is set to 185. By default, BT should support larger packets and I assume this figure is based on the IP/UDP overhead?

What about those values like 1280 for the IP MTU... ? Any chance to increase the throughput somehow?

P.S.
It appears modifying the values in the link controller setup has no effect. At least, there is no update of the settings. Can you confirm that or let me know how to use it?

Is it possible to use this framework to write small program to sync Fitbit device on Windows PC? (Such program would take Fitbit user ID and password and sync Fitbit device with Fitbit server.)

Hello when I try to build the project I get this error

~/golden-gate$ inv native.build
loading initial cache file xp/config/profiles/linux.cmake
-- Build backend = Ninja, not normalizing paths
-- Captured Git commit info: hash=85fe404* branch=master commit_count=75
-- Configuring done
CMake Error at mbedtls/CMakeLists.txt:38 (add_library):
  No SOURCES given to target: gg-mbedtls

Hi there,

https://datatracker.ietf.org/doc/html/rfc7252#section-9.1.3.3

with regard to the above RFC, I was wondering if there are plans to support X.509 in the near future? Is there any draft code available maybe?

We're thinking on using Golden Gate as our communication framework for all of our IoT devices.
After we did a successful (and enjoyable!) first pass of the framework we're following up with a second & deeper research and found some doubts that we would like to address before proceeding.

We saw different signals that make us think that the framework is prepared for one-shot communications like those that usually happen with BLE & mobile apps. In our use case we've those kind of scenarios but we also have others like ones that happen through long-lived connections. These persistent connections require special attention at different layers like reestablish DTLS session without doing handshakes (pion/dtls#264) and/or have streaming requests (CoAP observe pattern, for example).

The framework cannot be more pluggable than it is right now with GG_Data{Sink,Source}, components we could use to cook our own layers and overtake the limitations but I would like to know which is the short-term vision on this matter in Golden Gate? Do you have use cases like the one I mentioned?

Hi,

This isn't an issue, but a couple questions I have when it comes to using golden gate.

(1) I find a datagram socket in gg_sockets.h, but no stream socket. Is it correct that golden gate currently only provides a datagram socket?

(2) The GATT link MTU seems to be much smaller than IP MTU. But I do find a IP packet assembler code in golden gate. Do I understand it right that if I program against the datagram socket interface in gg_socket, I only need to worry about IP MTU, not the GATT MTU or the L2CAP MTU?

(3) For NIP, GG_NipDatagramSocket_Create seems ignoring the max_datagram_size parameter. Then what type of MTU limit should I expect from NIP?

(4) I see both Android and iOS_Macos profile files set Lwip as the default socket module. May I know why isn't BSD used as default since you also provide a BSD port?

(5) Should I expect the C APIs in general in Golden Gate to work on both iOS and Android?

Thanks!