slimevr / slimevr-rust Goto Github PK

We need to add hardware timestamps to UnfusedData. We may want to be generic on the timestamp type, so that we can represent when things are hardware timestamps vs software timestamps.

Fusion algos will need this, as several can use a dt to improve performance: https://github.com/jmagnuson/ahrs-rs#field_access

Looks like the atomic trap handler in esp-hal is being a ඞඞsussy bakaඞඞ

Blocked by esp-rs/riscv-atomic-emulation-trap#3 and also knurling-rs/defmt#597
Related: embassy-rs/embassy#995

Options for workarounds:

• proceed as normal (i.e. compile to -imac and use trap handler) but compile just embassy with opt-level = 0
• Switch to -imc and fork defmt-rtt to use atomic-polyfill instead of core::sync::atomic

Regardless of which approach to use, I think that I should be eschewing core::sync::atomic in favor of atomic-polyfill in my code, to avoid this issue on any platforms without an atomic trap handler

I've been tracking my time spent debugging this (just for science) and I think I've spent about 24hrs over the last 4 days or so

We don't want to drop any data from the imu, and we can use the fifo to accomplish this. However, I don't believe that we are actually doing this currently with the driver for the bmi160.

There is a decent chance that someone will want to try running the skeletal model on a microcontroller without an OS. We should support a no_std + alloc environment.

Current blockers:

• we use thiserror and that doesn't support no_std. Could try snafu instead.
• petgraph doesn't support no_std 🥲

https://github.com/esp-rs/esp-idf-sys

The reason to use esp-idf is that we can easily port program from std, although it is only a fraction of.

#116 added a basic imu implementation which reads the gyro velocity from the bmi. However, we should be using rotational position rather than velocity, so we need to integrate or use sensor fusion.

Also, we need to test that the math of the conversion on velocity and to euler angles actually made sense.

Need to add feature-gated support for the BNO08X

This library looks like a promising one to build on.

I would like an idea of what performance improvements optimizations can bring. Some ideas for optimizations:

• #73
• Switch index type to u8

Before I actually implement these I'd really like to have some benchmarks in place to see what the impact of the change is. A good library to use would probably be criterion.

#187 Introduced a logfile. It will always erase the last log at program start. Since past errors are helpful when debugging users' errors, we should instead rotate the logfile. This means that there is some maximum file size (or number of lines) that we permit, and once we hit that maximum size, we truncate old logs until we are back at the maximum size.

In order to remove the cli popup, we should add logging to a file. The file should be a rotating log ideally, to avoid it becoming too big.

It seems radium is trying to use atomics in ESP32 and its failing to build

Need to add suppot for the ICM20948 IMU

This crate is 1 week old but looks promising

We need to come up with a BLE based protocol, that we can use in the esp and nrf net-ble.

I had to remove the firmware from the cargo workspace, because it only compiles on RISC-V currently, and has its own cargo.toml. I wasn't sure how to do this in a better way, so for now I removed it from the cargo workspace.

I saw that rust-lang/cargo#5220 exists but seems to have no movement.

One solution is for me to have the main.rs have a cfg(platform) feature gate. And maybe I could feature gate the dependencies and then activate that feature when on the appropriate architecture?

Prior art: SlimeVR/SlimeVR-Feeder-App@6b9263f

The LSM6DS3 is an important IMU, because it is included in the seeed studio sense board. It has lower accel and gyro noise than bmi160.

We are allocating here, here and here.

We should be reusing the capacity of these data structures rather than dropping and reallocating them every time we solve. I was too tired to implement this initially, but eventually we should do it.

We can use defmt_bbq to allow us to log from defmt into a queue, and then pop from that queue and do something with some peripherals.

This would allow for example, an asynchronous task that logs using uart to read from the queue and act as the global logger.

We should add a new feature flag called mcu-nrf52840 to the existing mcu-* feature flag set. If active, we will not have any esp-hal or esp-wifi dependencies, and will only have nrf-hal related stuff.

To make matters easier, I suggest we start by using defmt-rtt as the global logger, and net-stubbed as the network transport. This will ensure that we only need to focus on getting the right setup done for the HAL, passing it to the imu driver, configuring the right feature flags, and setting up embassy.

We had changes in SlimeVR/SlimeVR-OpenVR-Driver#24 that switched the tracking space from raw to "standing". We need to account for this change in the overlay.

Need to add tests for that code, since its non-trivial.

ESP-WIFI bundles (for some reason) a global allocator. I think I'm going to need to submit a PR to remove it so the user can bring their own.

Technically they use the same one that we do, though.

esp-rs/esp-wifi-sys#70

Example: https://github.com/SlimeVR/SlimeVR-Rust/security/code-scanning/1

I have an idea that may help to better calibrate the proportions of the body to get the Slime in 1X1 calibration

The idea is to create an app within steam that will work exclusively for calibrating the position of the trackers.

This application is made for use with SteamVR, and when you enter it asks which trackers you will use with SlimeVR.
After selecting the desired types of trackers, squares representing the trackers will appear in front of the user.
The next step now is to grab the tracker within the application and drag it to where the corresponding tracker is in the body.
After doing this with all the trackers, the program will create a code containing the skeleton configuration that can be used in the SlimeVR software.

Need to add support for 9250 imu

I (intentionally) broke net-wifi in #193. We should fix it, and in the process use embassy_net for our wifi

defmt requires a global logger. The nrf52840 support defmt-rtt out of the box, but unlike the esp32c3, we don't have USB JTAG, so we would need to read the RTT via a hardware probe + JTAG or SWD. I would prefer to implement defmt::Logger using the nrf's usb serial, to make logging really easy and provide a good way to verify that the device is working.

There is some example code here that demonstrates how to set up the nrf52 for usb serial. We should take that code and adapt it for the firmware, and then implement a global logger using it.

Initiate discovery from client-side in initial handshake state, instead of waiting for server sent heartbeat.

The esp wifi implementation is bugged where you cannot broadcast udp after receiving udp, so this also helps workaround this bug. It also just makes more sense.

cargo embed starts a GDB server for us to connect to as a remote target to debug. We need to document how to accomplish this using lldb, and ideally also with vscode.

I don't currently know how to do this, so if anyone knows, please comment here or open a PR!

Users have been asking for the ability to see the skeleton overlay in front of you, as if you were looking in a mirror. We should implement the ability to turn this on and off in the Skeleton or something.

The arduino firmware uses i2cscan to select an i2c address. It would be nice if we could autodetect the I2C address based on a list of i2c address options.

Basic support was implemented here:

• #116
• #172

To improve support, we will need to do these:

• #196
• #195

@loucass003 added support for changing the overlay settings here. We need to implement the new protocol to respond to the GUI's requests.