V2 to V3 boot error (571) esp_core about arduino-esp32 HOT 9 OPEN

I turned core debug level to verbose: and it shows the partition info and some additional stats:

E (750) esp_core⸮f0WW�⸮e⸮+⸮�-'H⸮KW�$VW���V,W,⸮,ZY⸮-HV⸮-[Y⸮⸮
Calculated checksum='233d5ab2'
Image checksum='ffffffff'
[ 247][D][esp32-hal-cpu.c:264] setCpuFrequencyMhz(): PLL: 480 / 2 = 240 Mhz, APB: 80000000 Hz
[ 272][V][esp32-hal-periman.c:160] perimanSetPinBus(): Pin 3 successfully set to type UART_RX (2) with bus 0x3ffbedd0
[ 283][V][esp32-hal-periman.c:160] perimanSetPinBus(): Pin 1 successfully set to type UART_TX (3) with bus 0x3ffbedd0
=========== Before Setup Start ===========
Chip Info:

Model : ESP32
Package : D0WD-Q5
Revision : 100
Cores : 2
CPU Frequency : 240 MHz
XTAL Frequency : 40 MHz
Embedded Flash : No
Embedded PSRAM : No
2.4GHz WiFi : Yes
Classic BT : Yes
BT Low Energy : Yes
IEEE 802.15.4 : No

INTERNAL Memory Info:

Total Size : 316756 B ( 309.3 KB)
Free Bytes : 286164 B ( 279.5 KB)
Allocated Bytes : 23512 B ( 23.0 KB)
Minimum Free Bytes: 280548 B ( 274.0 KB)
Largest Free Block: 110580 B ( 108.0 KB)

Flash Info:

Chip Size : 16777216 B (16 MB)
Block Size : 65536 B ( 64.0 KB)
Sector Size : 4096 B ( 4.0 KB)
Page Size : 256 B ( 0.2 KB)
Bus Speed : 80 MHz
Bus Mode : QIO

Partitions Info:

            nvs : addr: 0x00009000, size:    20.0 KB, type: DATA, subtype: NVS
        otadata : addr: 0x0000E000, size:     8.0 KB, type: DATA, subtype: OTA
           app0 : addr: 0x00010000, size:  1920.0 KB, type:  APP, subtype: OTA_0
           app1 : addr: 0x001F0000, size:  1920.0 KB, type:  APP, subtype: OTA_1
         spiffs : addr: 0x003D0000, size:   128.0 KB, type: DATA, subtype: SPIFFS
       coredump : addr: 0x003F0000, size:    64.0 KB, type: DATA, subtype: COREDUMP

Software Info:

Compile Date/Time : Jul 3 2024 14:24:16
Compile Host OS : windows
ESP-IDF Version : v5.1.4-358-gbd2b9390ef-dirty
Arduino Version : 3.0.2

Board Info:

Arduino Board : ESP32_DEV
Arduino Variant : esp32

from arduino-esp32.

After much testing I have determined this is not a V3 error. I have narroed it down to something with the partition setup I think. Originally my device used spiffs and I flashed the device using esptool.exe with the following command:

esptool.exe --chip esp32 --port COM7 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 bootloader_qio_80m.bin 0x8000 Main.ino.partitions.bin 0xe000 boot_app0.bin 0x10000 myapp.bin 0x3d0000 Main.spiffs.bin

Now if I try to flash that with a firmware compiled with 2.0.17 using that same command line I get the core dump line but the chip continues to work.

If I erase the chip and flash the same firmware with esptool.exe removing the spiffs reference then I no longer get the dump.

esptool.exe --chip esp32 --port COM7 --baud 921600 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 80m --flash_size detect 0x1000 bootloader_qio_80m.bin 0x8000 Main.ino.partitions.bin 0xe000 boot_app0.bin 0x10000 myapp.bin

I am not sure if maybe the reserved space for spiffs changed over the different revisions? Once I flash firmware with my spiffs reservation causing the crash I continue to get the crash even if you remove the reference. The chips is stuck until you do an erase.

So my problem is how do I move forward. I no longer need spiffs in my project but my devices were flashed with that address reservation. The problem recreates if I try to OTA a new file to them.

Appreciate any advice! :^)

from arduino-esp32.

I have narrowed it down. I am still running 2.0.17 just as an example and it doesn't matter what code I have. If I flash the device with the original spiffs.bin it gives the error on startup. This spiffs file had 3 files in it. The file size on the old spiffs.bin shows 192kb or 196,608 bytes. I sort of remember back in 2020 when I created that file that it always created a 192K spiffs file. When I try to create a new spiffs file with the same 3 files I get the error:

[SPIFFS] data : C:\SPYDERROBOTICSDATA\spyderrobotics\FIRMWARE\arduino\SKETCHBOOK\SPYDERWEB_12\JUNK\Main\data
[SPIFFS] start : 3997696
[SPIFFS] size : 128
[SPIFFS] page : 256
[SPIFFS] block : 4096
/jquery.min.js
/spyderweb_logo_left.gif
/spyderweb_logo_right.gif
SPIFFS_write error(-10001): File system is full.

error adding file!

SPIFFS Create Failed!

If you select the 3 files in windows it shows file size to be 92.2KB (94,485 bytes).
If you remove one of the files it will create a new spiffs file.

If I flash with the new spiffs file on an erased chip (doesn't matter what program) everything is fine.
Trying to flash with the new spiffs file over the top of a chip that had the previous larger spiffs file does not correct the core dump error.

So in summary:
The partition scheme I am using is the Minimal Spiffs which shows 190KB. Was this ever 192K previously and that is why it generated the 192K spiffs file that I have?

I am thinking there was some difference with the partition scheme in the past which allowed that file to work and something in later revisions of the esp core that choke when trying to flash over the previous firmware.

Other than the error on boot the device appears to be working but I don't know if there are other effects yet.

Since I have many devices out there with this original configuration it would be nice to find a solution that can fix it via OTA. I did try flashing code that enabled spiffs and did a format. The format was successfull but it still had the core error on boot.

from arduino-esp32.

Haven't read everything in detail but it does seem like a partition mismatch between what is deployed and what is built using latter versions of core 2.0.x and the newer core 3.0.x.

Solution, if this is the case, is to build the new version using the partition table from what is deployed.

To find that table, a couple of options:

1. go back through the core v2.0.x releases and grab the version used to build the deployed software. In the ../esp32/hardware/esp32/[CORE_VERSION]/tools/partitions folder you'll find all the partition tables. Grab the default from there.
2. if you have an device with the old version, use esptool.py to extract the table from the flash of that device and then convert that to a .csv file.

Once you have the table in .csv format, make sure the file name is partitions.csv and place it in the same location as your sketch .ino file.

Then when the software is compiled, the build system will use that table.

Caveat:
Older versions of the arduino-esp 32 core had a smaller bootloader and the user part of the partition table started at 0x8000. With newer cores, the boot loader is larger and so the user part now starts at 0x9000.

Meaning, you may need to customize the old portion table to compensate, but give it a whirl and see.

Docs on esp tool.py, converting to and from .bin and .csv formats, and partition tables in general is over on the espressif ESP-IDF site.

There are other things that might come in to play but one step at a time.

from arduino-esp32.

I was able to extract the partitions.csv from my original. After more testing with 3.0.2 installed and a basic program I get the core dump message if I have the old partition.csv in my sketchbook folder. If I remove the partitions.csv from my folder, erase the chip, and compile/flash, with the same selected partition setup I no longer get the core dump message. So somewhere along the line the esp32 core changed breaking compatibility with my partition setup. I looked at the difference in the two files and attached an image

The green partition setup is what 3.0.2 creates and the yellow is what my original project partition setup is. What I notice is the addition of the coredump partition but also the spiffs file size changes. Keep in mind this these were built with MINIMAL SPIFFS 1.9MB app with OTA and 190K spiffs. So what I think is happening is the coredump error on boot is because there is no coredump partition in my partitions.csv. But what also concerns me is the spiffs size is different between my old one. I am not great with conversions but with the new (green) partition setup is it actuatlly only giving 131K of space for spiffs and not the 192K? Seems like it cut down spiff space to make room for the coredump partition. But it still shows 190K in the description. Is there a way to prevent arduino from creating the coredump partition?

from arduino-esp32.

So in thinking about this the coredump error is because the coredump partition doesn't exist. Which in itself doesn't seem to be a problem because my code still is at the correct positions in flash. The challenge I guess is how can you update a device via OTA with code written in 3.0.2 to a device that does not have a coredump partition? OTA with the main .bin file isn't going to change partition structure correct? This is why it seems I just need the ability to disable the use of coredump to flash.

from arduino-esp32.

I found this config CONFIG_ESP_COREDUMP_TO_FLASH_OR_UART that has the option of

None (ESP_COREDUMP_ENABLE_TO_NONE)

I think this would eliminate my error but I am not sure if this is possible in the arduino core?

These was also CONFIG_ESP_COREDUMP_CHECK_BOOT that shows the default as:

Yes (enabled) if ESP_COREDUMP_ENABLE_TO_FLASH

Maybe this check system also needs to be disabled?

from arduino-esp32.

I don't believe the core dump partition is mandatory.

Does the code run if you use the table without that partition?

Didn't go back through everything, but what message are you seeing? If it's only a warning then no problem.

from arduino-esp32.

Yes. The code works. I just get a error out the serial on boot "E (571) esp_core⸮3�յ⸮}⸮⸮⸮͡⸮��⸮ɕ⸮"յ⸮⸮"⸮х⸮�⸮⸮⸮⸮⸮2⸮⸮⸮⸮⸮⸮jR4�⸮kV�+�W��⸮,ZY⸮⸮⸮][O⸮⸮⸮&YMX⸮⸮'" Functionally I could just ignore it as the error doesn't stop the code from running. It would just be nice to clean it up and also get and explanation why it seems the spiffs allocation does not match. If I am correct on that then that should be addressed because you can't actually use 190K as shown in the description if there is only 131,072 available per the partitions file that 3.0.2 is creating.

from arduino-esp32.