ESP-BOX lite freezes about willow HOT 41 CLOSED

...and what is the best way to make these parameter changes, going back to the ./utils.sh config command, followed by the build erase-flash and flash?

from willow.

Also - the middle button does not wake the device always. The only way to get out of this 'sleep state' is to press reset.

Please let me know if this is normal also?

UPDATE: It is not freezing, It's oprator error!!!
One should not wait after saying the wake-word. I am so used to Alexa and Google to wait for the device to wake up before the command, but not with WIllow. One seems to say the wake-word followed by the command.

from willow.

Hah, thanks for the update! I tested ESP BOX Lite again and was about to respond that I can't reproduce this. Those are the always the most "fun" issues to work with.

I should add we don't currently support the buttons on the front of the Lite. We're still trying to methodically design the UI on the device itself and we don't want to assign button functionality before we have a UI/UX that's coherent in the long term. That said it won't be long.

from willow.

Device has very erratic response - 1/10 times it would work. Sometimes afer wake-word it turns onto screen "Say Command" and stuck there. Other times it does not even respond to wake word.
Restarting and power-off/on does not seem to help either.
Will flash it once more and see if it behaves better.
Just FYI - I used a Raspberry Pi4 as my host device to generate and flash the firmware from - I do not think that would make a difference?
Not using local WIS, but online host.

from willow.

We weren't planning initially on supporting ESP BOX Lite but with the overwhelming demand people started buying them so we worked in support. It's received much less testing than ESP BOX and it's on our list to really put it through the paces. However, we have a test framework we use with loops of samples, etc and we haven't seen anything like what you're describing...

Building and flashing from Raspberry Pi is something we've never tested and I hadn't even considered it until this comment. I have to imagine it would lead to some issues... Can you able to build and flash from something x86_64 based?

from willow.

Yes. I do have an older Intel NUC -i3 laying around that I can deploy Debian on and do another flash-test.
Al the compiles and linking seems to went fine on the Raspberry Pi4, wiithout any errors.
Will keep you updated.

from willow.

Please do, you just seem to be experiencing instability we haven't seen since the first week of development so there have to be some other issues at play here...

from willow.

ok. I managed to get Ubuntu installed on an older NUC box. I restarted and are trying to flash, but this is the result:
[ 1479.461248] usb 1-1: new full-speed USB device number 7 using xhci_hcd
[ 1479.610594] usb 1-1: New USB device found, idVendor=303a, idProduct=1001, bcdDevice= 1.01
[ 1479.610615] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1479.610623] usb 1-1: Product: USB JTAG/serial debug unit
[ 1479.610629] usb 1-1: Manufacturer: Espressif
[ 1479.610635] usb 1-1: SerialNumber: 7C:DF:A1:FF:38:CC
[ 1479.614450] cdc_acm 1-1:1.0: ttyACM0: USB ACM device
[ 1482.290028] usb 1-1: USB disconnect, device number 7
[ 1492.657072] usb 1-1: new full-speed USB device number 8 using xhci_hcd
[ 1492.806376] usb 1-1: New USB device found, idVendor=303a, idProduct=1001, bcdDevice= 1.01
[ 1492.806394] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 1492.806401] usb 1-1: Product: USB JTAG/serial debug unit
[ 1492.806406] usb 1-1: Manufacturer: Espressif
[ 1492.806411] usb 1-1: SerialNumber: 7C:DF:A1:FF:38:CC
[ 1492.810556] cdc_acm 1-1:1.0: ttyACM0: USB ACM device
bert@NUC-Ubuntu:/willow$ export PORT=/dev/ttyACM0
bert@NUC-Ubuntu:/willow$ export port=/dev/ttyACM0
bert@NUC-Ubuntu:~/willow$ sudo ./utils.sh erase-flash
Using venv for esptool
esptool.py v4.5.1
Serial port

A fatal error occurred: Could not open , the port doesn't exist.

I exported both port and PORT with the device name (and it's found) - but I still get this error. And yes, I have disconnected and re-connected a few times.

What am I missing here?

Thanks

from willow.

PORT should be uppercase. Can you send the output of:

id
ls -l /dev/ttyACM0

from willow.

Sure:

`
bert@NUC-Ubuntu:~/willow$ id
uid=1000(bert) gid=1000(bert) groups=1000(bert),4(adm),24(cdrom),27(sudo),30(dip),46(plugdev),122(lpadmin),134(lxd),135(sambashare)

bert@NUC-Ubuntu:~/willow$ ls -al /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 May 26 14:30 /dev/ttyACM0

`

from willow.

You need to either run the flash commands with sudo -or- add your self to the dialout group:

usermod -a -G dialout bert

Then logout and login again

from willow.

I am running it with sudo (as I discovered it early on during the build process). Look at the output from three posts ago - I use sudo ./utils.sh .....I just added myself to this group and the result is still the same

from willow.

Sorry, missed that.

If you check dmesg do you see a lot of disconnect/reconnect messages for the device? Is the device connected directly to the NUC? Is there anything, anything at all you think might be unique about your setup/configuration?

I'm beginning to suspect your device may be defective.

from willow.

It looks like the group "dialout" does not exist:
bert@NUC-Ubuntu:~/willow$ groups
bert adm cdrom sudo dip plugdev lpadmin lxd sambashare
`

from willow.

It does. From your system:

bert@NUC-Ubuntu:~/willow$ ls -al /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 May 26 14:30 /dev/ttyACM0

The dialout group has write permission to the device and resolves to name, so it must exist.

from willow.

no - there is nothing unique about this setup, accept that I did a brand new install of Ubuntu 2204 onto this NUC box as used this instead of the Raspberry pi 4 I used before on your request. The only reason why there is a lot of connects/disconnects is that I initiates it. I am not sure how to exactly place it in bootloader mode? Disconnect power, press and hold the boot button (top one) add power, press and hold the reset button, let go of the reset button and let go of the boot button. That is when all these disconnects, reconnects happens and the screens stays blank.
When I use the other button combination (bottom, power, top, release top and then bottom) - the device boots normal. That tells me it's not bootloader mode. Is my assumption correct?

from willow.

oh, ok. I just thought that since i do not see the group "dialout" listed under the group command results, that this group might not exist.

from willow.

That means you are not in the group. groups shows you the groups you are in, not all of the groups on the system. You need to add yourself to the group, log out, then back in.

from willow.

I added myself to the group with:
sudo usermod -a -G dialout bert
logged out and back in again but it stil does not show in id command results.

from willow.

This is driving me crazy!.
I am not winning at all on the Ubuntu box. I do not know what I am doing wrong.

I am back on the raspberry-pi4 that I used earlier in the week. but the results are the same:
sudo dmesg:
....
[ 2064.161833] cdc_acm 1-1.3:1.0: ttyACM0: USB ACM device
[ 2065.762238] usb 1-1.3: USB disconnect, device number 7
[ 2068.095735] usb 1-1.3: new full-speed USB device number 8 using xhci_hcd
[ 2068.203091] usb 1-1.3: New USB device found, idVendor=303a, idProduct=1001, bcdDevice= 1.01
[ 2068.203218] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2068.203236] usb 1-1.3: Product: USB JTAG/serial debug unit
[ 2068.203250] usb 1-1.3: Manufacturer: Espressif
[ 2068.203263] usb 1-1.3: SerialNumber: 7C:DF:A1:FF:38:CC
[ 2068.210802] cdc_acm 1-1.3:1.0: ttyACM0: USB ACM device

pi@piWillow:/willow $ export PORT=/dev/ttyACM0
pi@piWillow:/willow $ sudo ./utils.sh erase-flash
You need to define the PORT environment variable to do serial stuff - exiting
pi@piWillow:/willow $ ./utils.sh erase-flash
Creating venv for esptool
Installing esptool...
.
.
esptool.py v4.5.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:ff:38:cc
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 9.5s
Hard resetting via RTS pin...
Flash erased. You will need to reflash.
./utils.sh: line 158: flags/erase-flash: Permission denied
sudo dmesg:
.....
[ 2334.986584] cdc_acm 1-1.3:1.0: ttyACM0: USB ACM device
[ 2335.816483] usb 1-1.3: USB disconnect, device number 12
[ 2338.695507] usb 1-1.3: new full-speed USB device number 13 using xhci_hcd
[ 2338.802446] usb 1-1.3: New USB device found, idVendor=303a, idProduct=1001, bcdDevice= 1.01
[ 2338.802462] usb 1-1.3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 2338.802468] usb 1-1.3: Product: USB JTAG/serial debug unit
[ 2338.802473] usb 1-1.3: Manufacturer: Espressif
[ 2338.802477] usb 1-1.3: SerialNumber: 7C:DF:A1:FF:38:CC
[ 2338.809821] cdc_acm 1-1.3:1.0: ttyACM0: USB ACM device
pi@piWillow:/willow $ export PORT=/dev/ttyACM0
pi@piWillow:/willow $ ls -al /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 May 26 16:00 /dev/ttyACM0
pi@piWillow:/willow $ ./utils.sh erase-flash
Using venv for esptool
esptool.py v4.5.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:ff:38:cc
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 2.0s
Hard resetting via RTS pin...
Flash erased. You will need to reflash.
./utils.sh: line 158: flags/erase-flash: Permission denied
pi@piWillow:/willow $ sudo ./utils.sh erase-flash
You need to define the PORT environment variable to do serial stuff - exiting
pi@piWillow:/willow $ echo $PORT
/dev/ttyACM0
pi@piWillow:~/willow $ id
uid=1000(pi) gid=1000(pi) groups=1000(pi),4(adm),20(dialout),24(cdrom),27(sudo),29(audio),44(video),46(plugdev),60(games),100(users),104(input),106(render),108(netdev),117(lpadmin),997(gpio),998(i2c),999(spi)

What the hell am I missing here.
My id (on the raspberry pi has access to dialout - by default.
When I run utils.sh withoug sudo, it works, but gets a permission denied in the last step - line 158.
When I run it wih sudo, it tells me the PORT env variable is not defined.

This is SO frustrating.

from willow.

I looked that the utils.sh code......
(sorry - I have no idea how code-source management works on github, so I do appologize if I took your code and modified it).But the exported env variable is NOT being picked up in the utils.sh code.
I added these two lines in the code just before the check_port() toutine:
export PORT=/dev/ttyACM0
echo $PORT

And guess what???? - it works.
I manged to erase-flash and flash it on the rpi4:
pi@piWillow:~/willow $ sudo ./utils.sh flash
_/dev/ttyACM0
Using venv for esptool
esptool esp32s3 -p /dev/ttyACM0 -b 2000000 --before default_reset --after hard_reset write_flash --flash_mode dio --flash_freq 80m --flash_size 16MB 0x0 bootloader/bootloader.bin 0x10000 willow.bin 0x8000 partition_table/partition-table.bin 0x210000 srmodels/srmodels.bin 0x710000 audio.bin
esptool.py v4.5.1
Serial port /dev/ttyACM0
Connecting...
Chip is ESP32-S3 (revision v0.1)
Features: WiFi, BLE
Crystal is 40MHz
MAC: 7c:df:a1:ff:38:cc
Uploading stub...
Configuring flash size...
Flash will be erased from 0x00000000 to 0x00005fff...
Flash will be erased from 0x00010000 to 0x00191fff...
Flash will be erased from 0x00008000 to 0x00008fff...
Flash will be erased from 0x00210000 to 0x005f3fff...
Flash will be erased from 0x00710000 to 0x0078ffff...
Compressed 22256 bytes to 13958...
Wrote 22256 bytes (13958 compressed) at 0x00000000 in 0.3 seconds (effective 517.6 kbit/s)...
Hash of data verified.
Compressed 1580976 bytes to 1017607...
Wrote 1580976 bytes (1017607 compressed) at 0x00010000 in 17.4 seconds (effective 727.4 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 133...
Wrote 3072 bytes (133 compressed) at 0x00008000 in 0.0 seconds (effective 546.9 kbit/s)...
Hash of data verified.
Compressed 4078356 bytes to 3230404...
Wrote 4078356 bytes (3230404 compressed) at 0x00210000 in 49.1 seconds (effective 664.4 kbit/s)...
Hash of data verified.
Compressed 524288 bytes to 26959...
Wrote 524288 bytes (26959 compressed) at 0x00710000 in 2.6 seconds (effective 1605.4 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
[tio 16:20:50] tio v1.32
[tio 16:20:50] Press ctrl-t q to quit
[tio 16:20:50] Connected_

from willow.

The "sleep" condition is back. I can issue one command and have to reset the device to do the next

from willow.

I did the same "code modification" on the Ubuntu box, and it works now there also. However, something is still up witht he BOX-lite and is falls asleep and no matter how many times I send the wake-word - it just ignores me.
High voice tone, low voice tone, shouting at it - just sleeping.

from willow.

I'll take a long hard look and do more testing with my Lites this weekend and get back to you - this is VERY strange.

from willow.

So I've been doing a few sporadic tests with an ESP BOX Lite over the past few hours and it's behaving exactly as it should.

We don't currently support the buttons or control of the backlight but wake, clean speech, commands, etc is working fine here. We'll be getting to these things soon enough but for now all of the audio, WiFi, etc functionality works just as well as the ESP BOX.

With all of your flashing issues and other strange things going on you may want to wait a month or so for us to fully support the ESP BOX Lite and have a user-friendly flashing and configuration process.

from willow.

I ran 1000 loops of our test harness against an ESP BOX Lite yesterday with zero failures. Can you confirm you're still experiencing this or elaborate on your issue?

from willow.

no, exactly same issues.
I rebuilded a Ubuntu box from scratch and re-installed the latest willow as of this morning.
Same issues. Does not recognise the wake-word. Also (as mentioned before) - I get the "I2S: register I2S object to platform failed" message.

The screen totally turns off now each time it goes to sleep and not responding.

from willow.

UPDATE:

The device is not freezing - it cannot recognize to my normal speaking voice.
When I lower my voice significatly - it works, The screen wakes up and it recognizes the wake-word and respond via Home Assistant just fine.

So, how do I get it to recognize my regular speaking voice?
Is there a set of recordings one can make for sample voice sounds during setup?

It is annoying that I have to lower my voice all the time, as it is not accurate and therefor difficult to re-produce.

from willow.

Try playing with these options: https://github.com/toverainc/willow/blob/main/main/Kconfig.projbuild#L277-L298

from willow.

Thanks.
What should I make these " WILLOW_WAKE_DET_MODE_*" values, lower or higher.
Is there documentation someplace that describe these in detail?

Thanks

from willow.

Is there documentation someplace that describe these in detail?

https://github.com/espressif/esp-sr/blob/master/include/esp32s3/esp_wn_iface.h#L17-L27

from willow.

Another thing I'd like to see is testing with Alexa as wake word. I know a lot of people don't like it but if nothing else it helps to separate any audio issues (the params @stintel mentioned) and pronunciation.

Hi ESP is actually a pretty bad wake word and we're finding that some people can't quite nail it, I suspect because of the requirement to clearly annunciate E-S-P which is pretty awkward.

from willow.

I do have multiple ALEXA devices around in my house. I will disable these, redo the installation with ALEXA and test that. Yes, I agree and really wish we can change the "Hi E.S.P." as it is a real pain to pick up the frequency of my voice.
I have 'up-ped' the frequency of the wakeword detection as suggested earlier. It is better, but still not perfect.

I will respond back after I have tested the ALEXA wake-word, with updates.

Thanks for the help.

from willow.

Which WILLOW_WAKE_DET_MODE did you select?

Thanks for testing this, I know it's kind of a pain currently with the build process, etc but it's impossible for us to know results with the endless variety of speakers so all of this testing with different speakers, environments, etc is very helpful!

Let us know how things go with Alexa, while not ideal we generally find it to be more reliable. It's this kind of data that we will incorporate in the training process for future wake words.

Thanks again!

from willow.

Since my voice is a little "higher pitched" I selected 95.
I redo the config with Alexa, disabled the "real alexa" devices and I must tell you - I had NO issues with the ESP lite picking up and respond to the command. Works like a charm.
I also received two ordered ESP-box devices today. One is already setup with wake-word Alexa and that works perfectly also. No problems.
My suggestion is that we really put the E.S.P. wake-word on the back-burner and come up with something different and easier to understand. Maybe even Homie or Assist (or assistant?

So far I am impressed with nof using ESP anymore.

from willow.

Thank you, this is great feedback!

We have seen with testing that the more aggressive "95" configurations eliminate a lot of wake detection issues - it will likely be the default shortly. One question - which channel (1, 2, 3) variant of "95" are you using? I personally have had excellent results with three channel "95" (the meaning of channel here is a complicated one) across both "Hi ESP" and "Alexa".

An alternative Willow specific wakeword is absolutely on the roadmap and we have several issues and discussions across the willow and willow-inference-server repos regarding this.

What I can tell you is that we will not directly fund a Home Assistant specific wake word but we are open to getting community feedback to lead crowd funding campaigns for additional wake words (at pass-through cost) as long as they meet our basic quality requirements:

• Three to four syllables (to prevent false wake activation).
• Uncommon in everyday speech/media (to prevent false activation).
• Consistent pronunciation across accents, languages, and different speakers (to prevent failure to wake).

If anyone wants to commission ANY wake word we won't take any steps to prevent it and we'd likely allow for users to upload their own commissioned and trained wake words for use with Willow but we're not keen on the variety of issues that would likely flood in with wakewords that are fundamentally poor choices.

from willow.

I use the 2channel variant of 95. I have not tried the 3channel yet, but will and report back. But so far I am much more happier with Alexa than ESP.

from willow.

That's great to hear! If you can try three channel 95 with Hi ESP and Alexa that would be very helpful.

from willow.

from willow.

Great! Just want to clarify one thing - multinet mode runs 100% on the ESP BOX itself. There is no additional server or anything like it.

from willow.

from willow.