ambiqai / neuralspot Goto Github PK

PDM audio, no options or instrumentation, no RPC, all platforms.

Instead of encoding, can we stream raw audio over BLE? If so, NNSE w/no codec might be possible on AP3.

• refactor to use C instead of H files in example
• rename to 'kitchen_sink' - update documents everywhere to reflect this

Neuralspot is coming along nicely @carloshmorales!

Here's a few little gotchas I ran into when trying to utilize my own nest for speech enhancement (most of which I've just manually worked around for the time being):

1. nest isn't relocatable as the dependencies under the ../make directory are not available from the <nest_dir>/Makefile
2. passing a BINDIR value that is outside the nest subtree doesn't work. May just want to document this somewhere? Possibly even add a check to ensure conformance?
3. Setting AUDIO_DEBUG=1 uncovered a lot of missing headers/code, most notably SEGGER_RTT. Maybe it makes sense to treat this as another extern module? I've manually extracted 7.70a from the main Jlink SDK to move forward with what I need. Also need to add and compile the .c code that comes with it.
4. Some AUDADC_* references were missing

• Main readme
• blog
• AS readme
• multiplatform blog

It would be nice to include the targets' SVD files and copied into the generated nest. This will make it easier to flash and debug.

A number of Python package dependencies are missing to run AutoDeploy. eRPC url is also malformed which silently fails and causes pip to fallback to latest eRPC from PyPi.

• tensorflow
• tabulate
• tqdm
• pydantic
• pydantic-argparse

For AutoDeploy, it would be nice to leverage static type hints for the various functions, especially those that use Pydantic models.

AutoDeploy CLI should expose a verbosity flag to control amount of logs, data pushed to stdout and stderr. E.g. only dump layer statistics when verbosity is high, Capture output of make commands when they fail.

On commit 296b185, I am unable to run ns_autodeploy.

From fresh install, I run the following

make PLATFORM=apollo4p_evb
cd tools/
python -m ns_autodeploy \
--tflite-filename model.tflite \
--model-name my model \
--cpu-mode 192 --runs 2 \
--arena-size-scratch-buffer-padding 10 \
--max-arena-size 80 \
--verbosity 4 

1. If I add --measure-power the script fails as the params for ns_perf don't match ns_autodeploy (missing joulescope and a bunch of others).

2. If neuralSPOT is not built with NS_MLPROFILE then the minimal example fails due to the following bug in the template and in some of the examples:

#ifdef NS_MLPROFILE
    ns_timer_config_t *tickTimer;
    ns_perf_mac_count_t *mac_estimates; ///< Optional, from tflm_profiler tool
#else
    void *tickTimer;
    void *mac_estimate; ///< Optional, from tflm_profiler tool
#endif

-> mac_estimate should be mac_estimates

As for ns_perf, when I run the following it hangs with the following message dumped INFO: handle_cmd_gpi_req s/gpi/+/!req. I'm on Mac M1 with GUI Joulescope UI 1.0.62, Joulescope driver 1.4.10, and python joulescope==1.1.13 and and pyjoulescope driver is 1.4.10. I do pre-launch Joulescope app and quit (green LED on scope). I also have Joulescope GPIO hooked up to EVB according to docs (PIN22 -> IN0, PIN23 -> IN1, PIN24 -> OUT0, GND -> GND).

make PLATFORM=apollo4p_evb
cd tools/
python -m ns_perf \
--cpu-mode "NS_MAXIMUM_PERF" \
--joulescope \
--tflite-filename model.tflite \
--max-arena-size 100 \
--arena-size-scratch-buffer-padding 20 \
--runs-power 100 \
--model-name modelperf \
--verbosity 4  

If I make local changes to get ns_autodeploy to work it does run completely so I don't think there is any issue with arena size or model architecture.

Instead of real-time SE, record a few seconds and clean/xmit when a button is pressed.

NNID model grabs CPU for too long - modify to suspend so that AP3 can run this without timing out BLE.

On certain models, the reported arena size by TFLM is insufficient and will result in subsequent AllocateTensors call to fail. This will cause AutoDeploy to hang when then trying to feed input stimulus.

For insufficient tensor arena, we might want to provide additional arena padding CLI argument.
For latter issue, AutoDeploy should verify TFLM model initialization is successful before attempting to send inputs.

-Platform: apollo4p
-os : ubuntu
I tried the basic_tf_stub example on ubuntu for checking whether PRC is working.
I followed the instruction // #define RPC_ENABLED in basic_tf_stub.cc and uncomment it and compile it. It seems not working.

1. NS_USB1_PRESENT below seems not existing.
   

   neuralSPOT/examples/basic_tf_stub/src/basic_tf_stub.cc

   Lines 15 to 17 in 696a2cd

   	#ifdef NS_USB1_PRESENT
   	// #define RPC_ENABLED
   	#endif

2. After I did make deploy, I can not find the ttyACM0 or ttyACM1. Usually, I can find it as below

poyupaulchen@paul:~/Documents/esc/evb$ ls -la /dev/serial/by-id
total 0
drwxr-xr-x 2 root root 80 Feb 28 17:26 .
drwxr-xr-x 4 root root 80 Feb 28 16:42 ..
lrwxrwxrwx 1 root root 13 Feb 28 16:42 usb-SEGGER_J-Link_000900030197-if00 -> ../../ttyACM0
lrwxrwxrwx 1 root root 13 Feb 28 17:26 usb-TinyUSB_TinyUSB_Device_123456-if00 -> ../../ttyACM1