Comments (11)
stephen99scott
commented on August 10, 2024
@adambujak @harneetMac Here's my first stab at the packet structure. 40 represents the number of samples per gesture. Specifying the max/fixed count makes it easier to populate the arrays with data.
syntax = "proto2";
message DataPacket {
repeated sint32 quaternion_w = 1 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated sint32 quaternion_x = 2 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated sint32 quaternion_y = 3 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated sint32 quaternion_z = 4 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 flex_thumb = 5 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 flex_index = 6 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 flex_middle = 7 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 flex_ring = 8 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 flex_little = 9 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch1 = 10 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch2 = 11 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch3 = 12 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch4 = 13 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch5 = 14 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch6 = 15 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch7 = 16 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch8 = 17 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch9 = 18 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch10 = 19 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
repeated uint32 touch11 = 20 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
}
from signbuddy.
stephen99scott
commented on August 10, 2024
We could pad the message with a header/footer at transmission with something like (where 0x16 and 0x27 are arbitrary values):
{
uint8_t SYNC_BEGIN 0x16
uint16_t SIZE
DataPacket
uint8_t SYNC_END 0x27
}
from signbuddy.
adambujak
commented on August 10, 2024
if we're framing it ourselves i think we should add a crc
from signbuddy.
stephen99scott
commented on August 10, 2024
Yeah a crc would be good to have. Also, it might make more sense to structure the data like this with a nested repeated message:
message DataPacket {
repeated Sample data = 1 [(nanopb).max_count = 40, (nanopb).fixed_count = true];
}
message Sample {
required sint32 quaternion_w = 1;
required sint32 quaternion_x = 2;
required sint32 quaternion_y = 3;
required sint32 quaternion_z = 4;
required uint32 flex_thumb = 5;
required uint32 flex_index = 6;
required uint32 flex_middle = 7;
required uint32 flex_ring = 8;
required uint32 flex_little = 9;
required uint32 touch1 = 10;
required uint32 touch2 = 11;
required uint32 touch3 = 12;
required uint32 touch4 = 13;
required uint32 touch5 = 14;
required uint32 touch6 = 15;
required uint32 touch7 = 16;
required uint32 touch8 = 17;
required uint32 touch9 = 18;
required uint32 touch10 = 19;
required uint32 touch11 = 20;
}
from signbuddy.
harneetMac
commented on August 10, 2024
FYI, quaternion data is sint16 (signed 2 bytes).
from signbuddy.
stephen99scott
commented on August 10, 2024
Protobuf doesn't support int16 as far as I'm aware
from signbuddy.
harneetMac
commented on August 10, 2024
Also, maybe you read this already. Google's website on protobuff warns users from using "required" field.
from signbuddy.
harneetMac
commented on August 10, 2024
Protobuf doesn't support int16 as far as I'm aware
yeah you are right
from signbuddy.
stephen99scott
commented on August 10, 2024
I think they don't recommend required fields due to potential backwards compatibility issues if you change your packet structure. That's mostly a concern if you have an application with many users and you want to make sure users with old versions are still supported by the server in the future. For our case it makes sense for all fields to be required
from signbuddy.
stephen99scott
commented on August 10, 2024
@adambujak @harneetMac What do you think of this structure? GestureData is only used on the Android side to construct the packet with all the GestureData. I was thinking including the letter in the data could make it easier for us for training. I.e. we can save a bunch of delimited protobuf packets to a file which we can copy to a desktop to train with. Also the sample packets include the sample id to ensure that the samples are placed in the correct order in the gesture data.
message GestureData {
required string letter = 1;
repeated Sample samples = 2;
}
message Sample {
required uint32 sample_id = 1;
message IMUData {
required sint32 quaternion_w = 1;
required sint32 quaternion_x = 2;
required sint32 quaternion_y = 3;
required sint32 quaternion_z = 4;
required sint32 lin_acc_x = 5;
required sint32 lin_acc_y = 6;
required sint32 lin_acc_z = 7;
}
required IMUData imuData = 2;
message FlexData {
required uint32 flex_thumb = 1;
required uint32 flex_index = 2;
required uint32 flex_middle = 3;
required uint32 flex_ring = 4;
required uint32 flex_little = 5;
}
required FlexData flexData = 3;
message TouchData {
required uint32 touch1 = 1;
required uint32 touch2 = 2;
required uint32 touch3 = 3;
required uint32 touch4 = 4;
required uint32 touch5 = 5;
required uint32 touch6 = 6;
required uint32 touch7 = 7;
required uint32 touch8 = 8;
required uint32 touch9 = 9;
required uint32 touch10 = 10;
required uint32 touch11 = 11;
}
required TouchData touchData = 4;
}
We would transmit Sample packets to the Android app padded like this:
{
uint8_t size = sizeof(sample) + sizeof(CRC)
Sample sample
int32_t crc = crc(sample)
}
from signbuddy.
stephen99scott
commented on August 10, 2024
Switching the touch sensors to bool brings the maximum encoded size down to 100 bytes:
message GestureData {
required string letter = 1;
repeated Sample samples = 2;
}
message Sample {
required uint32 sample_id = 1;
message IMUData {
required sint32 eul_p = 1;
required sint32 eul_r = 2;
required sint32 eul_y = 3;
required sint32 lin_acc_x = 4;
required sint32 lin_acc_y = 5;
required sint32 lin_acc_z = 6;
}
required IMUData imuData = 2;
message FlexData {
required uint32 flex_thumb = 1;
required uint32 flex_index = 2;
required uint32 flex_middle = 3;
required uint32 flex_ring = 4;
required uint32 flex_little = 5;
}
required FlexData flexData = 3;
message TouchData {
required bool touch1 = 1;
required bool touch2 = 2;
required bool touch3 = 3;
required bool touch4 = 4;
required bool touch5 = 5;
required bool touch6 = 6;
required bool touch7 = 7;
required bool touch8 = 8;
required bool touch9 = 9;
required bool touch10 = 10;
required bool touch11 = 11;
}
required TouchData touchData = 4;
}
from signbuddy.
Related Issues (20)
- integrate bno055 library into imu module
- support sending packets to android app
- create packet generator/encoder/decoder
- create communication layer between stm and ble module HOT 1
- get imu working
- Add COMMS task to firmware to transmit/receive data from the BLE module
- investigate why i2c reads from imu are so slow HOT 4
- Add interrupt support and collection task for ADC
- Deserialize protobuf packet in Android app
- Save gesture protobuf to training data file on Android app
- Add protobuf support for Python
- Import training data into Python HOT 3
- Quiz mode
- Lesson mode
- Switch IMU to relative mode
- Collect quaternion instead of Euler angles
- Calibrate flex sensors HOT 1
- Change GPIO pin for LED
- Export sklearn SVM model to Android (Java) using sklearn-porter
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from signbuddy.